{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook, I will publish some code examples for training AlexNet, using Keras and Theano.\n",
    "\n",
    "**Software Requirements :- **\n",
    "1. Theano - Follow the instructions available here http://deeplearning.net/software/theano/install.html#install\n",
    "2. Keras - Follow the instructions available here https://keras.io/#installation\n",
    "\n",
    "**We will acheive 3 objectives - **\n",
    "1. Establish the baseline performance of AlexNet on our classification task\n",
    "2. Train a basic ANN/SVM as a classifier, using the features extracted from a pre-trained AlexNet model\n",
    "3. Fine-Tune the layers of the AlexNet model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using Theano backend.\n",
      "WARNING (theano.gof.compilelock): Overriding existing lock by dead process '22535' (I am process '24494')\n",
      "Using gpu device 1: Tesla K20m (CNMeM is disabled, cuDNN 5103)\n",
      "/home/rahulduggal2608/anaconda2/lib/python2.7/site-packages/theano/sandbox/cuda/__init__.py:600: UserWarning: Your cuDNN version is more recent than the one Theano officially supports. If you see any problems, try updating Theano or downgrading cuDNN to version 5.\n",
      "  warnings.warn(warn)\n"
     ]
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "from keras.preprocessing.image import ImageDataGenerator\n",
    "from keras.optimizers import SGD\n",
    "\n",
    "from alexnet_base import *\n",
    "from utils import *"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Define some variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "batch_size = 16\n",
    "input_size = (3,227,227)\n",
    "nb_classes = 2\n",
    "mean_flag = True # if False, then the mean subtraction layer is not prepended"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the cell below, we define generators which are neat ways to perform realtime data augmentation. We use only basic data augmentation such as shear, zoom and flipping."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found 2000 images belonging to 2 classes.\n",
      "Found 800 images belonging to 2 classes.\n"
     ]
    }
   ],
   "source": [
    "#code ported from https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html\n",
    "\n",
    "# this is the augmentation configuration we will use for training\n",
    "train_datagen = ImageDataGenerator(\n",
    "        shear_range=0.2,\n",
    "        zoom_range=0.2,\n",
    "        horizontal_flip=True)\n",
    "\n",
    "test_datagen = ImageDataGenerator()\n",
    "                                  \n",
    "\n",
    "train_generator = train_datagen.flow_from_directory(\n",
    "        '../Data/Train',  \n",
    "        batch_size=batch_size,\n",
    "        shuffle=True,\n",
    "        target_size=input_size[1:],\n",
    "        class_mode='categorical')  \n",
    "\n",
    "validation_generator = test_datagen.flow_from_directory(\n",
    "        '../Data/Test',  \n",
    "        batch_size=batch_size,\n",
    "        target_size=input_size[1:],\n",
    "        shuffle=True,\n",
    "        class_mode='categorical')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Task 1 : Training AlexNet from scratch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ontain the AlexNet model object. The get_alexnet function abstracts two things\n",
    "    1. Implements real-time mean subtraction through prepending a custom layer\n",
    "    2. Specifies the initialization parameter of each layer to he_normal"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/rahulduggal2608/anaconda2/lib/python2.7/site-packages/keras/layers/core.py:577: UserWarning: `output_shape` argument not specified for layer mean_subtraction and cannot be automatically inferred with the Theano backend. Defaulting to output shape `(None, 3, 227, 227)` (same as input shape). If the expected output shape is different, specify it via the `output_shape` argument.\n",
      "  .format(self.name, input_shape))\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "____________________________________________________________________________________________________\n",
      "Layer (type)                     Output Shape          Param #     Connected to                     \n",
      "====================================================================================================\n",
      "input_1 (InputLayer)             (None, 3, 227, 227)   0                                            \n",
      "____________________________________________________________________________________________________\n",
      "mean_subtraction (Lambda)        (None, 3, 227, 227)   0           input_1[0][0]                    \n",
      "____________________________________________________________________________________________________\n",
      "conv_1 (Convolution2D)           (None, 96, 55, 55)    34944       mean_subtraction[0][0]           \n",
      "____________________________________________________________________________________________________\n",
      "maxpooling2d_1 (MaxPooling2D)    (None, 96, 27, 27)    0           conv_1[0][0]                     \n",
      "____________________________________________________________________________________________________\n",
      "convpool_1 (Lambda)              (None, 96, 27, 27)    0           maxpooling2d_1[0][0]             \n",
      "____________________________________________________________________________________________________\n",
      "zeropadding2d_1 (ZeroPadding2D)  (None, 96, 31, 31)    0           convpool_1[0][0]                 \n",
      "____________________________________________________________________________________________________\n",
      "lambda_1 (Lambda)                (None, 48, 31, 31)    0           zeropadding2d_1[0][0]            \n",
      "____________________________________________________________________________________________________\n",
      "lambda_2 (Lambda)                (None, 48, 31, 31)    0           zeropadding2d_1[0][0]            \n",
      "____________________________________________________________________________________________________\n",
      "conv_2_1 (Convolution2D)         (None, 128, 27, 27)   153728      lambda_1[0][0]                   \n",
      "____________________________________________________________________________________________________\n",
      "conv_2_2 (Convolution2D)         (None, 128, 27, 27)   153728      lambda_2[0][0]                   \n",
      "____________________________________________________________________________________________________\n",
      "conv_2 (Merge)                   (None, 256, 27, 27)   0           conv_2_1[0][0]                   \n",
      "                                                                   conv_2_2[0][0]                   \n",
      "____________________________________________________________________________________________________\n",
      "maxpooling2d_2 (MaxPooling2D)    (None, 256, 13, 13)   0           conv_2[0][0]                     \n",
      "____________________________________________________________________________________________________\n",
      "lambda_3 (Lambda)                (None, 256, 13, 13)   0           maxpooling2d_2[0][0]             \n",
      "____________________________________________________________________________________________________\n",
      "zeropadding2d_2 (ZeroPadding2D)  (None, 256, 15, 15)   0           lambda_3[0][0]                   \n",
      "____________________________________________________________________________________________________\n",
      "conv_3 (Convolution2D)           (None, 384, 13, 13)   885120      zeropadding2d_2[0][0]            \n",
      "____________________________________________________________________________________________________\n",
      "zeropadding2d_3 (ZeroPadding2D)  (None, 384, 15, 15)   0           conv_3[0][0]                     \n",
      "____________________________________________________________________________________________________\n",
      "lambda_4 (Lambda)                (None, 192, 15, 15)   0           zeropadding2d_3[0][0]            \n",
      "____________________________________________________________________________________________________\n",
      "lambda_5 (Lambda)                (None, 192, 15, 15)   0           zeropadding2d_3[0][0]            \n",
      "____________________________________________________________________________________________________\n",
      "conv_4_1 (Convolution2D)         (None, 192, 13, 13)   331968      lambda_4[0][0]                   \n",
      "____________________________________________________________________________________________________\n",
      "conv_4_2 (Convolution2D)         (None, 192, 13, 13)   331968      lambda_5[0][0]                   \n",
      "____________________________________________________________________________________________________\n",
      "conv_4 (Merge)                   (None, 384, 13, 13)   0           conv_4_1[0][0]                   \n",
      "                                                                   conv_4_2[0][0]                   \n",
      "____________________________________________________________________________________________________\n",
      "zeropadding2d_4 (ZeroPadding2D)  (None, 384, 15, 15)   0           conv_4[0][0]                     \n",
      "____________________________________________________________________________________________________\n",
      "lambda_6 (Lambda)                (None, 192, 15, 15)   0           zeropadding2d_4[0][0]            \n",
      "____________________________________________________________________________________________________\n",
      "lambda_7 (Lambda)                (None, 192, 15, 15)   0           zeropadding2d_4[0][0]            \n",
      "____________________________________________________________________________________________________\n",
      "conv_5_1 (Convolution2D)         (None, 128, 13, 13)   221312      lambda_6[0][0]                   \n",
      "____________________________________________________________________________________________________\n",
      "conv_5_2 (Convolution2D)         (None, 128, 13, 13)   221312      lambda_7[0][0]                   \n",
      "____________________________________________________________________________________________________\n",
      "conv_5 (Merge)                   (None, 256, 13, 13)   0           conv_5_1[0][0]                   \n",
      "                                                                   conv_5_2[0][0]                   \n",
      "____________________________________________________________________________________________________\n",
      "convpool_5 (MaxPooling2D)        (None, 256, 6, 6)     0           conv_5[0][0]                     \n",
      "____________________________________________________________________________________________________\n",
      "flatten (Flatten)                (None, 9216)          0           convpool_5[0][0]                 \n",
      "____________________________________________________________________________________________________\n",
      "dense_1 (Dense)                  (None, 4096)          37752832    flatten[0][0]                    \n",
      "____________________________________________________________________________________________________\n",
      "dropout_1 (Dropout)              (None, 4096)          0           dense_1[0][0]                    \n",
      "____________________________________________________________________________________________________\n",
      "dense_2 (Dense)                  (None, 4096)          16781312    dropout_1[0][0]                  \n",
      "____________________________________________________________________________________________________\n",
      "dropout_2 (Dropout)              (None, 4096)          0           dense_2[0][0]                    \n",
      "____________________________________________________________________________________________________\n",
      "dense_3_new (Dense)              (None, 2)             8194        dropout_2[0][0]                  \n",
      "____________________________________________________________________________________________________\n",
      "softmax (Activation)             (None, 2)             0           dense_3_new[0][0]                \n",
      "====================================================================================================\n",
      "Total params: 56,876,418\n",
      "Trainable params: 56,876,418\n",
      "Non-trainable params: 0\n",
      "____________________________________________________________________________________________________\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "alexnet = get_alexnet(input_size,nb_classes,mean_flag)\n",
    "    \n",
    "print alexnet.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We train the AlexNet CNN using Stochastic Gradient descent for 80 epochs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/80\n",
      "2000/2000 [==============================] - 45s - loss: 0.2601 - acc: 0.5410 - val_loss: 0.2468 - val_acc: 0.5513\n",
      "Epoch 2/80\n",
      "2000/2000 [==============================] - 42s - loss: 0.2386 - acc: 0.5940 - val_loss: 0.2248 - val_acc: 0.6312\n",
      "Epoch 3/80\n",
      "2000/2000 [==============================] - 42s - loss: 0.2315 - acc: 0.6200 - val_loss: 0.2173 - val_acc: 0.6450\n",
      "Epoch 4/80\n",
      "2000/2000 [==============================] - 43s - loss: 0.2202 - acc: 0.6530 - val_loss: 0.2127 - val_acc: 0.6637\n",
      "Epoch 5/80\n",
      "2000/2000 [==============================] - 44s - loss: 0.2122 - acc: 0.6775 - val_loss: 0.2001 - val_acc: 0.6975\n",
      "Epoch 6/80\n",
      "2000/2000 [==============================] - 44s - loss: 0.1983 - acc: 0.6940 - val_loss: 0.1892 - val_acc: 0.7200\n",
      "Epoch 7/80\n",
      "2000/2000 [==============================] - 42s - loss: 0.1905 - acc: 0.7175 - val_loss: 0.1857 - val_acc: 0.7225\n",
      "Epoch 8/80\n",
      "2000/2000 [==============================] - 41s - loss: 0.1812 - acc: 0.7360 - val_loss: 0.1794 - val_acc: 0.7488\n",
      "Epoch 9/80\n",
      "2000/2000 [==============================] - 42s - loss: 0.1724 - acc: 0.7565 - val_loss: 0.1805 - val_acc: 0.7225\n",
      "Epoch 10/80\n",
      "2000/2000 [==============================] - 41s - loss: 0.1638 - acc: 0.7630 - val_loss: 0.1603 - val_acc: 0.7725\n",
      "Epoch 11/80\n",
      "2000/2000 [==============================] - 41s - loss: 0.1592 - acc: 0.7635 - val_loss: 0.1603 - val_acc: 0.7638\n",
      "Epoch 12/80\n",
      "2000/2000 [==============================] - 43s - loss: 0.1468 - acc: 0.7920 - val_loss: 0.1533 - val_acc: 0.7800\n",
      "Epoch 13/80\n",
      "2000/2000 [==============================] - 42s - loss: 0.1428 - acc: 0.7945 - val_loss: 0.1613 - val_acc: 0.7662\n",
      "Epoch 14/80\n",
      "2000/2000 [==============================] - 41s - loss: 0.1426 - acc: 0.7890 - val_loss: 0.1548 - val_acc: 0.7638\n",
      "Epoch 15/80\n",
      "2000/2000 [==============================] - 42s - loss: 0.1333 - acc: 0.8165 - val_loss: 0.1426 - val_acc: 0.8025\n",
      "Epoch 16/80\n",
      "2000/2000 [==============================] - 41s - loss: 0.1312 - acc: 0.8175 - val_loss: 0.1675 - val_acc: 0.7638\n",
      "Epoch 17/80\n",
      "2000/2000 [==============================] - 42s - loss: 0.1290 - acc: 0.8305 - val_loss: 0.1524 - val_acc: 0.7775\n",
      "Epoch 18/80\n",
      "2000/2000 [==============================] - 41s - loss: 0.1258 - acc: 0.8240 - val_loss: 0.1506 - val_acc: 0.7937\n",
      "Epoch 19/80\n",
      "2000/2000 [==============================] - 41s - loss: 0.1192 - acc: 0.8400 - val_loss: 0.1470 - val_acc: 0.8025\n",
      "Epoch 20/80\n",
      "2000/2000 [==============================] - 42s - loss: 0.1122 - acc: 0.8435 - val_loss: 0.1473 - val_acc: 0.7950\n",
      "Epoch 21/80\n",
      "2000/2000 [==============================] - 42s - loss: 0.1091 - acc: 0.8490 - val_loss: 0.1370 - val_acc: 0.8025\n",
      "Epoch 22/80\n",
      "2000/2000 [==============================] - 41s - loss: 0.1032 - acc: 0.8650 - val_loss: 0.1448 - val_acc: 0.8063\n",
      "Epoch 23/80\n",
      "2000/2000 [==============================] - 41s - loss: 0.1008 - acc: 0.8675 - val_loss: 0.1471 - val_acc: 0.8037\n",
      "Epoch 24/80\n",
      "2000/2000 [==============================] - 40s - loss: 0.1028 - acc: 0.8625 - val_loss: 0.1416 - val_acc: 0.8113\n",
      "Epoch 25/80\n",
      "2000/2000 [==============================] - 41s - loss: 0.0923 - acc: 0.8790 - val_loss: 0.1430 - val_acc: 0.8050\n",
      "Epoch 26/80\n",
      "2000/2000 [==============================] - 41s - loss: 0.0934 - acc: 0.8750 - val_loss: 0.1573 - val_acc: 0.7825\n",
      "Epoch 27/80\n",
      "2000/2000 [==============================] - 42s - loss: 0.0972 - acc: 0.8675 - val_loss: 0.1356 - val_acc: 0.8287\n",
      "Epoch 28/80\n",
      "2000/2000 [==============================] - 41s - loss: 0.0894 - acc: 0.8770 - val_loss: 0.1382 - val_acc: 0.7963\n",
      "Epoch 29/80\n",
      "2000/2000 [==============================] - 41s - loss: 0.0856 - acc: 0.8900 - val_loss: 0.1394 - val_acc: 0.8100\n",
      "Epoch 30/80\n",
      "2000/2000 [==============================] - 41s - loss: 0.0843 - acc: 0.8855 - val_loss: 0.1382 - val_acc: 0.8137\n",
      "Epoch 31/80\n",
      "2000/2000 [==============================] - 43s - loss: 0.0806 - acc: 0.8920 - val_loss: 0.1487 - val_acc: 0.7937\n",
      "Epoch 32/80\n",
      "2000/2000 [==============================] - 41s - loss: 0.0733 - acc: 0.9055 - val_loss: 0.1388 - val_acc: 0.8087\n",
      "Epoch 33/80\n",
      "2000/2000 [==============================] - 41s - loss: 0.0780 - acc: 0.9030 - val_loss: 0.1540 - val_acc: 0.7950\n",
      "Epoch 34/80\n",
      "2000/2000 [==============================] - 41s - loss: 0.0730 - acc: 0.9030 - val_loss: 0.1447 - val_acc: 0.8113\n",
      "Epoch 35/80\n",
      "2000/2000 [==============================] - 41s - loss: 0.0687 - acc: 0.9105 - val_loss: 0.1404 - val_acc: 0.8275\n",
      "Epoch 36/80\n",
      "2000/2000 [==============================] - 41s - loss: 0.0711 - acc: 0.9070 - val_loss: 0.1356 - val_acc: 0.8213\n",
      "Epoch 37/80\n",
      "2000/2000 [==============================] - 42s - loss: 0.0676 - acc: 0.9075 - val_loss: 0.1418 - val_acc: 0.8137\n",
      "Epoch 38/80\n",
      "2000/2000 [==============================] - 41s - loss: 0.0612 - acc: 0.9130 - val_loss: 0.1353 - val_acc: 0.8250\n",
      "Epoch 39/80\n",
      "2000/2000 [==============================] - 43s - loss: 0.0626 - acc: 0.9145 - val_loss: 0.1295 - val_acc: 0.8187\n",
      "Epoch 40/80\n",
      "2000/2000 [==============================] - 43s - loss: 0.0590 - acc: 0.9235 - val_loss: 0.1322 - val_acc: 0.8263\n",
      "Epoch 41/80\n",
      "2000/2000 [==============================] - 42s - loss: 0.0516 - acc: 0.9330 - val_loss: 0.1362 - val_acc: 0.8187\n",
      "Epoch 42/80\n",
      "2000/2000 [==============================] - 42s - loss: 0.0542 - acc: 0.9315 - val_loss: 0.1627 - val_acc: 0.7975\n",
      "Epoch 43/80\n",
      "2000/2000 [==============================] - 41s - loss: 0.0490 - acc: 0.9375 - val_loss: 0.1469 - val_acc: 0.8150\n",
      "Epoch 44/80\n",
      "2000/2000 [==============================] - 41s - loss: 0.0560 - acc: 0.9225 - val_loss: 0.1579 - val_acc: 0.8087\n",
      "Epoch 45/80\n",
      "2000/2000 [==============================] - 42s - loss: 0.0495 - acc: 0.9375 - val_loss: 0.1374 - val_acc: 0.8163\n",
      "Epoch 46/80\n",
      "2000/2000 [==============================] - 43s - loss: 0.0604 - acc: 0.9205 - val_loss: 0.1418 - val_acc: 0.8063\n",
      "Epoch 47/80\n",
      "2000/2000 [==============================] - 42s - loss: 0.0470 - acc: 0.9395 - val_loss: 0.1443 - val_acc: 0.8150\n",
      "Epoch 48/80\n",
      "2000/2000 [==============================] - 42s - loss: 0.0471 - acc: 0.9375 - val_loss: 0.1450 - val_acc: 0.8150\n",
      "Epoch 49/80\n",
      "2000/2000 [==============================] - 41s - loss: 0.0378 - acc: 0.9500 - val_loss: 0.1303 - val_acc: 0.8263\n",
      "Epoch 50/80\n",
      "2000/2000 [==============================] - 41s - loss: 0.0465 - acc: 0.9400 - val_loss: 0.1500 - val_acc: 0.8075\n",
      "Epoch 51/80\n",
      "2000/2000 [==============================] - 43s - loss: 0.0423 - acc: 0.9460 - val_loss: 0.1381 - val_acc: 0.8350\n",
      "Epoch 52/80\n",
      "2000/2000 [==============================] - 43s - loss: 0.0404 - acc: 0.9480 - val_loss: 0.1342 - val_acc: 0.8175\n",
      "Epoch 53/80\n",
      "2000/2000 [==============================] - 42s - loss: 0.0402 - acc: 0.9475 - val_loss: 0.1280 - val_acc: 0.8325\n",
      "Epoch 54/80\n",
      "2000/2000 [==============================] - 41s - loss: 0.0357 - acc: 0.9535 - val_loss: 0.1591 - val_acc: 0.8075\n",
      "Epoch 55/80\n",
      "2000/2000 [==============================] - 41s - loss: 0.0339 - acc: 0.9565 - val_loss: 0.1524 - val_acc: 0.8113\n",
      "Epoch 56/80\n",
      "2000/2000 [==============================] - 41s - loss: 0.0421 - acc: 0.9475 - val_loss: 0.1431 - val_acc: 0.8275\n",
      "Epoch 57/80\n",
      "2000/2000 [==============================] - 42s - loss: 0.0342 - acc: 0.9570 - val_loss: 0.1507 - val_acc: 0.8125\n",
      "Epoch 58/80\n",
      "2000/2000 [==============================] - 41s - loss: 0.0422 - acc: 0.9445 - val_loss: 0.1550 - val_acc: 0.8125\n",
      "Epoch 59/80\n",
      "2000/2000 [==============================] - 42s - loss: 0.0377 - acc: 0.9505 - val_loss: 0.1497 - val_acc: 0.8100\n",
      "Epoch 60/80\n",
      "2000/2000 [==============================] - 42s - loss: 0.0294 - acc: 0.9625 - val_loss: 0.1544 - val_acc: 0.8163\n",
      "Epoch 61/80\n",
      "2000/2000 [==============================] - 42s - loss: 0.0290 - acc: 0.9605 - val_loss: 0.1419 - val_acc: 0.8313\n",
      "Epoch 62/80\n",
      "2000/2000 [==============================] - 43s - loss: 0.0359 - acc: 0.9535 - val_loss: 0.1312 - val_acc: 0.8387\n",
      "Epoch 63/80\n",
      "2000/2000 [==============================] - 43s - loss: 0.0294 - acc: 0.9630 - val_loss: 0.1402 - val_acc: 0.8263\n",
      "Epoch 64/80\n",
      "2000/2000 [==============================] - 43s - loss: 0.0345 - acc: 0.9555 - val_loss: 0.1517 - val_acc: 0.8200\n",
      "Epoch 65/80\n",
      "2000/2000 [==============================] - 44s - loss: 0.0331 - acc: 0.9565 - val_loss: 0.1438 - val_acc: 0.8275\n",
      "Epoch 66/80\n",
      "2000/2000 [==============================] - 43s - loss: 0.0333 - acc: 0.9540 - val_loss: 0.1373 - val_acc: 0.8337\n",
      "Epoch 67/80\n",
      "2000/2000 [==============================] - 43s - loss: 0.0342 - acc: 0.9570 - val_loss: 0.1387 - val_acc: 0.8313\n",
      "Epoch 68/80\n",
      "2000/2000 [==============================] - 41s - loss: 0.0207 - acc: 0.9745 - val_loss: 0.1678 - val_acc: 0.8013\n",
      "Epoch 69/80\n",
      "2000/2000 [==============================] - 42s - loss: 0.0295 - acc: 0.9615 - val_loss: 0.1452 - val_acc: 0.8250\n",
      "Epoch 70/80\n",
      "2000/2000 [==============================] - 41s - loss: 0.0262 - acc: 0.9670 - val_loss: 0.1398 - val_acc: 0.8400\n",
      "Epoch 71/80\n",
      "2000/2000 [==============================] - 42s - loss: 0.0198 - acc: 0.9745 - val_loss: 0.1524 - val_acc: 0.8263\n",
      "Epoch 72/80\n",
      "2000/2000 [==============================] - 42s - loss: 0.0249 - acc: 0.9690 - val_loss: 0.1495 - val_acc: 0.8187\n",
      "Epoch 73/80\n",
      "2000/2000 [==============================] - 42s - loss: 0.0249 - acc: 0.9685 - val_loss: 0.1471 - val_acc: 0.8387\n",
      "Epoch 74/80\n",
      "2000/2000 [==============================] - 41s - loss: 0.0270 - acc: 0.9640 - val_loss: 0.1410 - val_acc: 0.8363\n",
      "Epoch 75/80\n",
      "2000/2000 [==============================] - 43s - loss: 0.0222 - acc: 0.9695 - val_loss: 0.1390 - val_acc: 0.8350\n",
      "Epoch 76/80\n",
      "2000/2000 [==============================] - 42s - loss: 0.0263 - acc: 0.9660 - val_loss: 0.1448 - val_acc: 0.8275\n",
      "Epoch 77/80\n",
      "2000/2000 [==============================] - 41s - loss: 0.0245 - acc: 0.9670 - val_loss: 0.1328 - val_acc: 0.8438\n",
      "Epoch 78/80\n",
      "2000/2000 [==============================] - 42s - loss: 0.0183 - acc: 0.9760 - val_loss: 0.1382 - val_acc: 0.8413\n",
      "Epoch 79/80\n",
      "2000/2000 [==============================] - 43s - loss: 0.0265 - acc: 0.9660 - val_loss: 0.1338 - val_acc: 0.8400\n",
      "Epoch 80/80\n",
      "2000/2000 [==============================] - 41s - loss: 0.0138 - acc: 0.9830 - val_loss: 0.1325 - val_acc: 0.8387\n"
     ]
    }
   ],
   "source": [
    "sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)\n",
    "alexnet.compile(loss='mse',\n",
    "              optimizer=sgd,\n",
    "              metrics=['accuracy'])\n",
    "\n",
    "history = alexnet.fit_generator(train_generator,\n",
    "                        samples_per_epoch=2000,\n",
    "                        validation_data=validation_generator,\n",
    "                        nb_val_samples=800,\n",
    "                        nb_epoch=80,\n",
    "                        verbose=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAakAAAEbCAYAAABgLnslAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnXd4FUXXwH8ngQAhlAAhhEAKvYQOoQmEjoiCoiiK2NDX\n3nsD9HttCHZfRSl2FBEsNIkYqkCQDgECIQFCSKhp9GS+P2Zzc0NuQvq9gfk9zz53d9qe3buzZ+fM\nmRlRSmEwGAwGgyvi5mwBDAaDwWDIC6OkDAaDweCyGCVlMBgMBpfFKCmDwWAwuCxGSRkMBoPBZTFK\nymAwGAwui1FSBpdHRP4WkbudLYfB4GqISKCIZIrIZfsuv2wv7FKISISIHBeRis6WxVmIiJ+IHChk\nnhkiclZEUqwtVUQ2lpaMhssbEdknIv2cLYcjROR5Efm/QubJtOpEit3v06Ulo8VlPdj1ilRSIhII\nXAVkAteV8bndy/J8l2AosLAI+d5WSlW3tmpKqQ4lLZjB4AJcAywoZB4FtLWrG9WVUu+WgmxXDFek\nkgLGAv8AM4E77SNEpLKITBaRWBE5ISLLRaSSFXeViKyywuNEZKwVnsMcJSJ3iMgKu+NMEXlQRHYD\nu62w90Vkv4gki0ikiFxll95NRF4UkT3Wl1ikiPiLyMcikuOBF5FfReSxiy9QRD4VkUkXhc0Tkcft\ngoZiVUIReU5EDlrnixKRvoW5oVYZWaaHe0Uk3tqesov3sK473jrXe/YtWREZLiIbrXsSLSKD7IoP\nEpGVlnyLRKSWlaeSiHwjIket/2WtiPgUVnaD62E9R9HWfztPRPzs4t4TkUTrWdksIq2s8KEist16\nTg6IyJMOyvWwnpVWdmF1ROSUiNSxjmsCTYF/RKS2iPxu5TkmIsvyE9vaHF3PeBGZLSKzLPnWi0hb\nu/gW1rvkhIhsFZFr7eLyfC9Z5xtjvZOSRORFu3xdrPdHsogkXPz+KBcopa64DYgG/gN0BM4BPnZx\nnwBLgXroP78bUBEIAFKAUYA74I3+YgL4G7jbrow7gOV2x5nAYqAGUMkKuxWoif5QeAJIADysuGeA\nzUAT67iNdb4uwEG7cmsDaUAdB9fYC4izO64JpAO+1nEF4AhQFWgG7LeLCwCC87h3M4DX8ogLtK71\nO6AyEAIkAf2s+NeA1ZbctYFVwEQrLhQ4aZfWD2hmd3+jgcZAJev4DSvuPuBXK1yADoCXs58xsxW4\nLu7L+s8vCu9nPZ/trPr3IbDMihsERALVrOPmds/uIaCHtV8DaJ/Heb8EXrc7fhBYYHd8M/Cdtf8G\n8KlVV92BnvlcTybQKI+48cBZ4HqrnKeAGGu/gvWMP2ft90W/b5paefN6L2XVuc8BD6AtcAZobuVb\nDdxm7XsCoc7+zwv9jDhbgDK/YG3mOwt4W8c7gMesfQFOASEO8j0PzMmjzIIoqT6XkOs40Mba3wkM\nyyPddqC/tf8Q8Ec+ZcYCV1n744Bwu7h+wBJrvzFwGOgPVLiEnDOA05a8J6zfGVZcVoVpapf+beAL\na38PMNgubhAQY+1/BkzO5/6+aHf8QNYLBbgLWJl178xWvjbyVlJfAm/ZHVe16m2A9QLfCXQF5KJ8\nscC9WAosn/P2B/bYHa8Extgdf233cp8IzAUaF+B6MtEfW/b1Y6AVNx5YbZdWgHigp/VeOnRRWd8D\nr17ivRQIZAB+dmFrgVHWfoR13trO/q+Lul2J5r6xwJ9KqRPW8Q9opQJQB/1FHuMgX0NgbzHOe9D+\nQESeFpEdVtP9BFDdOn/WuRzJALryjLH2xwDf5HPOH4HR1v6t6BZOFjZTn1JqL/A4MAFIFJHv7U0r\nDpiklKqllPK2fu+yi1PkvNY4oL61Xx/dYnMUd6n7e9hu/xTgZe1/g26lzrJMiG+Ja/X7GYpGffTz\nAYBSKh39wvdXSv0NfIxuXSSKyGcikvU8jET3JcVZprNueZT/N1DFMocFoltscwFERICBwCIr7Tvo\nZ/NPywT/3CVk73BR/VhiF2dzVFJai8Rb11rfPs4iDvBHvxcqk/c7ASDRbt++ftyDbmnutEzh11xC\ndpfjilJSIlIZba7rY9lnE9Av53Yi0gY4im4qN3aQ/QDQJI+i09FN6SzqOUhj88Cx+p+eAW60HmRv\ndNM+y5Z9IA8ZAL4Fhlu27BbAvDzSgVbAN4pIAPqrc45dnE1JASilZimleqG/zADeyqfc/BC0wski\nAG2CwfoNtIsLtIvL75rzRCl1QSn1ulKqNdADuBb9IWIo3+R4VkSkKtpEHA+glPpYKdUZaIV+CT9j\nhf+rlBoB+KDNwD85KlwplWnF3Yr+kPvDUoSgzeqxSqljVtp0pdTTSqnGaEerJy/RZ+uwT8rCVjcs\nZdjAutZD6LpiT4B1vfm9l/JFKbVXKXWrUsoHrWx/FpEqhS3HmVxRSgptC74AtER/ObWz9lcCY60v\nmxnAFNHu2W4i0s3q3P8O6C8iN4qIu4jUEpF2VrmbgBtEpIqINEF/veRHNeA8cMzqxH3VCsviS+B1\nqyxEpI2IeAMopeKB9egWxByl1Nm8TqKU2gQcs8pbpJRKscoLQvd/7bKOm4lIXxHxQPfRnUabLYrK\nK9a9aI02x82ywn8AXrY6qesAr5DdEpwG3GXJISJSX0SaXepEIhImIiGix4mkoe9rcWQ3lD0eoh1g\nsjZ39LNyl4i0tRwE3gD+UUrtF5HOIhIqIhXQz+oZIFNEKorIrSJSXSmVAaSiTWF58QO67+lWtGkt\ni6HA/KwDEblGRLIURCr6HVLUZ6yTiIywrvEJS/Y1aBNduog8KyIVRCQMGAb8YL2XpuP4vQT5KEUR\nuS3LGQRIRn8sl6/64Wx7Y1luaHfrdxyE34T+knFDN6unoE1WJ9A23Sxnh57oByoZ3RS/3QqvjTY5\nJQMr0HZk+z6pDOw6U63zTLPSxwNPo5vy/eziX7TCktEPcH27/LdZZfYuwDW/bKW9wS7sIeBDu+M2\n1jmS0V9tvwH18ihvBrpipVhbKpBkxWX1SY2zrusQ8JRd3krA+1Z4PPAelrOIFT8c7TCSgvaCzLLl\nLyWPPj/gFnT/RCra+eQ9wM3Zz5rZClwn91nPZ4b17GRgOeagnWL22D2T9a3wfnbPSRL6Q8cT7Uiw\nEP1hdtJ6prtf4vzRaAeNCnZhkUBHu+PHLTlT0ebqF/MpL0s5ptj9TrHixqNbbz9Y4f8C7ezytkS/\nb04C24Dr7OIq4eC9RHaflJtdWlt9se5NonW+rcC1zv7PC7uJdSGlgohMQ38NJCql2uaR5kPgarTJ\n7E6lv/4N+SAivYBvlFJBRcw/H/hIKbXokokLV24gWrFWVNqcYigGIjIErdTdgGlKqbcvir8OeB39\ncj8PPKGUWlWQvAbHiEhdYINSqkEplD0e7XxhzNGFoLTNfTOAwXlFisjV6D+tKdol/LNSlqfcYzXx\nHwO+KEYxf1tbaZCfPd5QQCzz5cfo+tMaGC0iLS5KFq6Uaqf0YOp70GbdguY1OKYG2jXc4CKUqpJS\nSq1EN03zYjjaWw2l1Fqghoj4lqZM5RnrRXMC8AU+KGo5Sql3VT59WcWk9JrmVxahQLRSKk4pdR7d\nrzfcPoFS6pTdoRfZfQ2XzGtwjFIqWin1o7PlMGRTwcnn9yen22W8FZboOPmVjVJqJ9mupS6HUioO\nPTDRUHwurhsH0conByIyAngT7c2W5V5coLyGskUpNdHZMpRHnK2kCoyImC90Q5milHJ506VSah4w\nzxrW8H/o8T0FxtQrQ1lT2HrlbBf0eHKOqWlghTmkJD1Gxo8f75JlGdlcozwXIJ6c42YuVTdWAo1E\nz2lY2Lwu+R+48vNhZCvaVhTKQknlOeEi2q00a5LWbsBJpZQx9RkM2g26iehJez3Qrva/2SewG7uD\niHREu/MfL0heg6G8UKrmPhH5HggDaovIfvQ4AQ/0jCBTlVILRM9avAftgn5X3qUZDFcOSqkMEXkY\n+JNsN/IoEfkPVv0BRoqeiT9rAPao/PI65UIMhmJSqkpKKXVrAdI8XJoy5EVYWJhLllXS5RnZyi9K\nj2NrflHY53b776CnuilQ3rLgSnk+jGxlR6kO5i1JRESVF1kN5R8RQZUDx4niYuqVoSwpSr0qN959\neREUFERcXNylE16hBAYGEhsb62wxDOUMU6/yxtSpsqXct6QszewEicoH5v4UnNRU+OknuOce05Iy\nz03emHtTdIpSr5ztgm4wuAwrVsC33zpbCoPBYI9RUgaDxd9/Q9/8VgkyGAxljlFShisWpcC+a8Eo\nKYPB9TBKylCuWLwY/v03d/jXX8Pu3YUrKyICGjeGuDg4eVLn79q1RMQ0GAwlhFFSLs4DDzzAf//7\nX2eL4TK8/TZMmpQzbPt2ePBBuO46SEkpeFn//S80aQKffALLl0O3buDhUbLyGlwTU6/KD8a7r5QJ\nDg5m2rRp9OvXzynnd/X7UxjOnoXataFCBTh0CDw9tclu8GC45hqIioIjR3SrqmrVnHlXr4YuXaCi\nteD2mjVw882wZAn06AE33ADBwfDCCzreePe59nPjzHrl6vfGlTHefeWMjIwMZ4tQroiMhObNoXNn\nWLhQh/3xBxw4oFtSH3wAp09DnTpQty78/rtOM3s2hIVpZXb0qE7z2mvw7LPQrBlcdRV8+aXpj7pc\nMPXq8sIoqVJk7Nix7N+/n2HDhlG9enUmTZqEm5sb06dPJzAwkP79+wMwatQo/Pz88Pb2JiwsjB07\ndtjKuOuuu3j11VcBWLZsGQ0bNmTKlCn4+vri7+/PzJkznXFpTmHZMq1sbrpJK57Dh7Vy+ugj3UKq\nVAkWLIBTp2DePD3e6aOPdJrVqyE0VCulOnV0q+zuu3W5jz8O1atDp05OvTxDATH16srCKKlS5Ouv\nvyYgIID58+eTkpLCqFGjAFi+fDk7d+5k8eLFAAwdOpS9e/eSlJREx44due222/Is8/Dhw6SmpnLo\n0CG+/PJLHnroIZKTk8vkekoCpSAhIXf48eOOHSLsiYiAPn20aW7RIq2s7r4bBgzImU5Em/B++UWb\n7yZN0q2vt96Cdeu0SfCvv6BKFZ0+LAz27s02BRpcG1OvrjBKct2R0ty0qLnJKzxnmuJvRSUoKEj9\n9ddfSimlYmNjlZubm4qNjc0z/YkTJ5SIqJSUFKWUUnfeead65ZVXlFJKRUREKE9PT5WRkWFLX7du\nXbV27dp8rr0YwpcCCxfq+zlsmFLr1mWHP/KIUrVqKXXsmON8584p5eWl1PHj+njAAKWuvlopu1vh\nkPT0oslp3TenP/elvRW1XpVEnSqv9crV6lR5oij16opoSZVEdSpJGjRoYNvPzMzk+eefp0mTJtSs\nWZPg4GBEhKNHjzrMW7t2bdzcsv82T09P0tLSSlbAUuSbb+Ddd7Wjw9Ch2gHi2DE900PfvjAxjwW2\n16/Xnnje3vr4++9hzhxwu8QT7OlZsvIbNCWlpkqSK7leXc6U+wlmXR2R3I4s9mHff/89v//+O0uX\nLiUgIIDk5GS8vb2zvnLLDadPw48/wpgx2vvu1CkYP167jGfV/dRUmD8f3n8ffHy0w8OTT0KbNjB8\nuDbLtWqllVdCgvbkGzYMLlzQjg19+mSfz8fHOddpcA2ulHplMEqq1KlXrx4xMTH069fP3sRiIzU1\nlUqVKuHt7U16ejovvPCCwwroymzcqJXTgQO6X+e22+Crr3SL6dproXdvnW7uXOjVK1vBvPSSVlDz\n52uX8Dp14M034f779aDa7dvhvfcgPV23oL76ynnXaHAtroR6ZdBcEeY+Z/L888/z+uuvU6tWLebM\nmZOroowdO5aAgAD8/f0JCQmhR48ehSrf2RVv1y4YNAhefFF73L3xhm75TJ6sW0GzZmWn/fZbuP32\n7GNPT5g6Fe64A1q31mH33AP79ul8GzfC6NFw333a5dzXt2yvzeC6XO71ypCNGcx7mVOc+7Nkie4H\nCg52HH/2LHTvrpXI/ffrPoYuXbTJLjpa9xt17Qrx8VrxdOum97O86lwZM5jX1Ku8MPem6FyRix4a\nSgel4D//0Urou++yw6Oi4H//0/Hx8RAYqNOBdv1+6SXtIj5njlZujRrpVtBrr8Hrr5cPBWUwGFwH\nY+4zOGTdOv27YIH2wFNKt5jCwnT/UOPGEBKiHRrsLSPDh2slNny4Ph49GsaOhfr19aBag8FgKAzG\n3HeZU9T78/jjWhklJmovO39/3X+0cmXh3LoPH9aDbufO1Y4R5QVj7jP1Ki/MvSk6RalXRkld5hTl\n/mRkQMOGen0lpfTcdiJaQTVvXkqCuhhGSZl6lRfm3hQd0ydlKBZffKHnuOvaFerVy1ZIV18NAwde\nOQrKYDC4DqYldZljf3+mTYOdO7PXY5o+Xc9j99xz+jgsTJv1wsPh+efhiSecI7MrYFpSpl7lhbk3\nRceY+wy5yLo/27bpGRtEdD+Tuzv0769XpP33Xzh/XvdBHTwI1arpdJeacuhyxigpU6/ywtybomPW\nkzI4ZOJE7WU3ebJuKa1bp1ewXbdOL5l+4gRs2qRdxmvW1ArsSlZQBoPBdTB9UlcAx4/r6YnuuEPP\nELFggfa6695dxy9bBnFx0LOnc+U0GAyGizHfy6VMcHAwS5cuLVYZX331Fb169Spy/g8+0NMViWgn\niAUL9ADboUOhXz/txbd6tV6DyWAoD7hCvTKUDUZJlQOUUiU2l1j37hATo2eEGDpUL49hlJThSqQk\n65WhFCnsAlTO2ijGoofO4vbbb1dubm7K09NTVatWTU2aNEmtWbNG9ejRQ9WsWVO1b99eRURE2NLP\nmDFDNWrUSFWrVk01atRIff/99yoqKkpVrlxZVahQQXl5eSlvb+9CyeDo/tx0k1KNGimVmanU+fNK\nVaumVN26+tigwQUWPQSGADuB3cBzDuJvBTZb20qgrV1crBW+EViXzznUuXN5Xr9L4ux65cr3xtUp\nSr1yuvIpsKDlUEkppVcQXbp0qVJKqfj4eFW7dm21aNEipZRS4eHhqnbt2uro0aMqPT1dVa9eXUVH\nRyullDp8+LDasWOHUkqpmTNnql69ehXp/I7uzx9/KDVpUvbxNdcoNWJEkYq/bHG2kkJbOfYAgUBF\nYBPQ4qI03YAaKluhrbGLiwG8C3AetW9fntfvsjizXrn6vXFlilKvrgjHCZlY/Ca9Gl90l1P938C3\n337LNddcw+DBgwHo378/nTt3ZsGCBYwcORJ3d3e2bt1KgwYN8PX1xbeU1qa45hq9ZXHffSW/Sqqh\n2IQC0UqpOAARmQUMR7esAFBKrbFLvwbwtzsWCmjOj4uDoKDCCVcSdQour3plKB2uCCVVnIpQksTF\nxfHTTz/x+++/A7qSXbhwgX79+uHp6cmPP/7IpEmTuPvuu7nqqqt49913aV4G0zxcd12pn8JQePyB\nA3bHB9GKKy/GAQvtjhWwREQygKlKqS/yyhgXV3jhXKVOgevWK0PJcEUoKWdi3zHbsGFDxo4dy+ef\nf+4w7cCBAxk4cCBnz57lpZde4r777mPZsmWmc9eQLyLSF7gLuMouuKdSKkFEfNDKKkoptdJR/hkz\nJhATo/fDwsIICwsrZYmLj6lX5YOIiAgiIiKKVYZRUqWM/TLXY8aMITQ0lJEjRzJgwADOnTvH2rVr\nadq0KRUqVGDNmjUMGDCAypUr4+XlhZs1otbX15eDBw9y/vx5Klas6OQrMpQR8UCA3XEDKywHItIW\nmAoMUUqdyApXSiVYv0dEZC66FeZQSTVuPIEJE0pO8LLA1KvywcUfPRMnTix8IYXtxHLWRjl1nPj1\n119VQECA8vb2VpMnT1br1q1Tffr0UbVq1VJ169ZVw4YNUwcOHFAJCQmqT58+qmbNmsrb21v17dtX\nRUVFKaWUOnfunBo2bJiqVauW8vHxKdT5Xf3+uCo433HCnWzHCQ+040TLi9IEANFAt4vCPQEva78q\nsAoYlMd51IABeV6/y+LMeuXq98aVKUq9KvW5+0RkCPA+uhN3mlLq7YviawLTgcbAaeBupdQOB+Uo\nR7KaebTyx9yfouEKc/dZdecDsuvOWyLyH3RFnyoiXwA3AHFoR4nzSqlQEQkG5qL7pSoA3yml3srj\nHKppU8Xu3bnCzXOTB+beFB2Xm2BWRNzQYzz6A4eASOAWpdROuzTvAKlKqddFpDnwiVJqgIOyjJIq\nAub+FA1XUFJlgYioypUV6ek552s0z03emHtTdFxxglmbG61S6jyQ5UZrTytgKYBSahcQZHX2GgyG\nMsDLC5KSnC2FweCY0lZSjtxo/S9KsxltskBEQtF29galLJfBYLAICLpQJDd0g6EscAXvvreAD0Rk\nA7AVPY1LhqOEE+xckMqLq6yhfFASrrLllbqND7F/fwBduzpbEoMhN6XdJ9UNmKCUGmIdP4/u9H07\nnzz7gDZKqbSLwk2fVBEw96doXEl9UqOfW0HzKlcxfnyOcPPc5IG5N0XHFfukIoEmIhIoIh7ALcBv\n9glEpIaIVLT27wWWXaygDAZD6dGsy35mzTJTYxlck1I19ymlMkTkYeBPst1oo+zdaIGWwFcikgls\nB+4pzDkCAwPNyPF8CAwMdLYIBhenUt39nD8P69dDly46zNSrvDF1qmwp9XFSJUVe5r7yztKlEBgI\njRvDtm16fafNm6F+fWdLdmVzJZn7HvjjAer9+ylHj8KHHzpbIsPljCua+wz5oBSMGwdhYRAdDXfe\nqVfQNQrKUJbEJccxZgzMmgXnzjlbGoMhJ0ZJOZENG8DdHV54Adq3h1q1tNIyGMqS/cn7adQImjeH\nxYudLY3BkBNXcEG/YvnpJ7jpJnjwQfD11Uu7m24AQ1mzP3k/ANdeC+Hh+tdgcBVMS8pJKAWzZ2sl\nBTBypDHzGZxDpsok+UwyPXvCqlXOlsZgyIlRUqXMpk2wZEnu8CxTX/v2ZS+TwWBPQI0A9ifvp1Mn\niIqC9HRnS2QwZGOUVCnz7rt6qfYZM3KGz5qlW1HGvGdwNllKqnJlaNcO1q1ztkQGQzZGSZUiSsFf\nf8Gvv8LEifDVVzo8PR1mzoS77nKqeAYDAAHVA4g9GQtgTH4Gl8MoqVIkKgoqVYIhQ+C33+DZZ+H4\ncZg+HXr3hqZNnS2hwQA9Gvbgz5g/9X4Po6QMroUZzFuKfPSR7pOaNk0fP/CAXrNn4UL47jvtzWdw\nTa6kwbwnT58k4P0AYh+L5VyKN82b648pN/MJayhhzGBeFyAuTs8cAdrU179/dtxrr2nl5OdnFJTB\ndahRuQb9g/szb+c8fH3Bxwe2b3e2VAaDxiipEuLsWXj1VejUCfr101MbLVum97Pw8YGvv4b33nOe\nnAaDI24JuYVZ22cB2hS9dKmTBTIYLIySKiFmzNAtp40btRIKCwN/f6hXL2e6666D0FCniGgw5Mmw\nZsNYe3AtSelJjBgBv/zibIkMBo2ZcaKEWLxYzxzRsCHcdhscPmyWPjCUHzwrejKs2TBmbprJowOf\n5fbb9TN88UeWwVDWGMeJEuDCBW3K27lTT29kKP9cSY4TWfVq59Gd9JrRi+0PbuexcXXp0wfuv9/J\nAhouK4zjhJOIjNTLbRgFZSjPtKjTgjFtxvDK0le48UaYM8fZEhkMRkmVCH/+CQMHOlsKg6H4vNrn\nVebtmkeDTltZtw6OHXO2RIYrHaOkSoAlS4ySMlweeFfxZmzbsYTv/42wMD0rusHgTIySKiYpKdrd\nvFcvZ0tiMJQMXfy7sD5hPZ0768HoBoMzMUqqmHz+uVZQVao4WxKDoWToUr8LkfGRtG9vlJTB+RgX\n9GKwbBlMngxr1jhbEoOh5AiqGcSZC2eo1zSBTZv8nC2O4QrHtKSKyJ49MHq0nkEiKMjZ0hgKQ0Zm\nBs/8+QzbkrY5WxSXREToXL8zCazn7Fk9XspgcBZGSRWBxYv1kgavvQaDBjlbGkNh2ZCwga+3fE2/\nr/px3+/38fe+vzmXcc7ZYuVCRIaIyE4R2S0izzmIv1VENlvbShFpW9C8l6Jz/c6sT9Amv82bi3sl\nBkPRMUqqkGzfDmPG6KXfx41ztjTln5SzKWxI2FDk/A/88QALohfkCl8Rt4Ij6Ucc5gmPCWd0yGii\nHorCv5o/z4U/R91JdRn500i+3vx1kWUpSUTEDfgYGAy0BkaLSIuLksUAvZVS7YD/A6YWIm++dKnf\nhfWH1pt+KYPTMUqqkMyfD7fcoifhNBSfZ5c8y/BZw8lUmYXOe/bCWb7d+i1fbf4qR7hSiht+uoFW\nn7Ziyj9TOH76eI748H3hDGg0gNqetRkfNp51965j18O7GN58uG3xPxcgFIhWSsUppc4Ds4Dh9gmU\nUmuUUsnW4RrAv6B5L0Xn+p2JPBRJu3bKtKQMTsUoqUJyJY+Jmrx6MjEnYkqsvE2HNzF351w8K3qy\nar/jlfbOXjjLqfOnHMat3L+S+tXqs3jPYs5cOGMLjzkRQyX3Siy/czmrD6wm6P0gwmaGcezUMU6d\nP8W6+HX0CeyToyxfL1/GthvLq31eLbHrKyb+wAG744NkKyFHjAMWFjFv7pNX96eiW0V8m+83LSmD\nUzFKqhCcPq09+cLCnC2Jc/h0/afMjZprO/58/eeknUsrUllKKR5f9DgTwyZyR7s7mLVtlsM0o34e\nRaMPGjF94/Rcra2FexZyW5vbaOPbhqX7steWWBu/lm4NutHSpyU/j/qZpGeSaFKrCeMjxrNy/0ra\n12tPtUrViiS3KyIifYG7gEL3PeVHF/8uHK+ylthYOOX4O8FgKHWMC/olSE+H//s/7SSxYgW0awfV\nqztbqrLnzIUz7Duxj2Vxy3iqx1OknE3h4YUP08i7EQMbF75puSxuGUnpSYzrOI64k3F0n9adD67+\ngApu2Y/k9I3TiTsZxy83/8KD8x/k5JmTPNn9SVv8gugFfH3913h5eDE3ai5Dmw4FYO3BtXT172pL\nV7lCZd4e8DYtP2lJ7MlYBgQPKMadKDPigQC74wZWWA4sZ4mpwBCl1InC5M1iwoQJtv2wsDDCrK+w\nng17subQSlq0GMXmzWahTkPhiYiIICIioniFKKXKxaZFLXtmzVIKlPrvf5V6+mmlJk4sXnnv/fOe\n+nn7z0UcLYOlAAAgAElEQVTOv+XwFpWZmVk8IYp4Xt9JvqrmWzXVhYwLas6OOYoJqCmrp+RK+++h\nfy9Z3pOLnlQTI7JvZugXoWrxnsW245jjMarOO3XU1sStSimlvvz3SzV27tgc8XUn1VUZmRlq7/G9\nqu6kuupCxgVbWctil+U654drPlRMQK2MW3lJ+aznzZnPuzuwBwgEPIBNQMuL0gQA0UC3wua1S5vn\nPVi9f7Vq/1l79eyzSr344iVvmcFwSYpSr4y57xLMng3jx+uFDH/4ofj9UWsOrmHVAcf9L5fi6Kmj\ntPusHTM3zXQYl0XUkSgeW/hY1kuoyHy49kPWH1qvyzwaRc+AntTzqseWxC0siF5Aa5/WucYaJaQm\n0Glqp0v2XS3Ys8DW8gEYHTKaLzd8aTt+dNGjPNX9KULqhgDQtHZToo9F2+IX7lnI1U2uxk3caOTd\niHpe9fg79m/OXjjLtqRtdPLrlOuc93e+nwl9JhDq7/qrTiqlMoCHgT+B7cAspVSUiPxHRO6zkr0C\n1AI+FZGNIrIuv7yFlaFT/U5EH4vm6hHJzJ5t1kczOAejpPIhLU07Sjz6KLzxhjb9delSvDIPpx0u\nsvPB0n1LCakbwrPhz7LvxD5b+D8H/sF/ij9bEreglOKRhY/w5cYvmbtzbj6l5c+p86d45e9XbH1F\nO4/upEXtFvQJ7ENEbAQLohfwTI9n2Jq0NUe+v/b9BcDC6IW5ysxi34l9HD99nI5+HW1h4zqOY0PC\nBuZGzWXxnsXsPLozh2mvaa2mRB/PVlJ/x/7NgEbZZrvnej7Hc+HPsSFhA81qN6OqR9Vc563oXpHx\nYeOp6F6xkHfDOSilFimlmiulmiql3rLCPldKTbX271VK1VZKdVRKdVBKheaXt7B4uHvQxb8Lp2qv\n5sIFM17K4ByMksqH+fO1Hb5WLT0mKioKKhSzF684Sio8Jpx7OtzDcz2f4455d9gcCb7c8CWd/Dpx\n+9zbmb1jNglpCfwy6hee/vPpHF5vhWHOjjna685q9e08upMWdbSS+nT9p1T1qMqIFiPYcWRHDoeG\nJTFL6BvUlwV7co9dymLhnoUMaTIEN8l+/Lw8vPjm+m94YP4DPLroUaYMmoKHu4ctvp5XPc5cOMPJ\nMycB7RnYoV4HW/zokNFUrlCZxxY9lqM/ylA8egX0YuWBFdx0k7YqGAxljVFS+TB7NowapfdFSmYp\n7SwlVRRTXHiMHt/zRLcnOHPhDN9u+Za0c2n8svMX5oyaQ3DNYG6dcyvvD36fwU0G065eO977570i\nyTlt4zTeGfAOWxK3cPr8aaKORmklFdSHPcf3MLTJUGpUrkFtz9q2sUVKKcJjwnl7wNusiFvB6fOn\nHZa9IHoBQ5sMzRXevWF37u98P01rNWVYs2E54kSEJrWaEH0smrRzacSnxNO8TvMc8R8M+YDIQ5F0\na9CtSNdsyE2vgF6s2K+V1E8/GZOfoewxSsoBc+bAnXdqU9+IESVX7unzpzlz4Qwe7h4cOeV4NoS8\niDkRw5kLZ2jl0wp3N3c+GPIBL/z1AtM3Tqd3YG/8qvnxxbVf8P6Q923edi/3epmvt+Q9g0LauTSO\nncq9qt2e43vYcWQHN4fcTGuf1qyNX8vuY7tpUacF9avVp0WdFjYlElI3hK2J2uS38+hOKrpVpHP9\nzrSv156I2IhcZZ+5cIblccsZ1NjxfFITwibw++jfEcm9wnSWyW9b0jZa1GmRwxMQ9ADU6ddNz6Xg\nDEWne8PubEzYSOt2Z8jIgC1bnC2R4UrDKKmLSEmB22+Hbt1g/Xpt6ispEtMT8fXypXGtxrlMfqfO\nn8q3dbVk7xIGNBpge3l3b9idfsH9eHLxk9zd/m4AfKr68HDow7Y8bX3bsj95P6lnUwHYmriVT9Z9\nYou/fe7t3D73dtvx7O2zGfb9MEb+NJIxbcfg4e5Bz4Y9mbVtFt6VvW1ji1bfvdrWHxTiE2Jznshq\n6YkIQ5sOZeGe3P1SU/6ZQu/A3nhX8c7zWh0pKLCU1LFotiRuoV29dg7T3NXhLup41smzbEPh8PLw\nonXd1izcs4BrroFFi5wtkeFKw4yTuohdu6BFC7j//uKXpZTK8cI9nHaYel71CKoZRMyJGJtZavWB\n1QyfNZxp103juubXOSwrfF841za7NkfYW/3fIvlMcg4vOXsqulekrW9bNh7eSO/A3ny9+Wsm/zOZ\n1nVbk5GZwabDmzh1/hS7ju6iQfUGPLLwEd4e8Da1PWvTL7gfAD0DenLPb/fQpX62x4i9gmnj24b5\n0fNtMo4OGQ3A0KZDGfTNIJRS1Khcg0dCH+FQ6iHeX/M+/973b2FvJaA9/MJjwklKT6Kdr2MlZSh5\n3hnwDrfMuYU3+4by9ccNeK5EhwwbDPljWlIXERUFLVsWv5z/Lv8vo34elaN1lKWkGns3Zu/xvQD8\ntus3RswaQZf6XVgRt8JhWRmZGSzdt5T+wf1zhPtX9+e30b/l663W2a8zkfGRgB5A+1zP57hz3p08\ntugxJg+azL0d7+XjdR/z2frP6BXYizva38GwZsPwrOgJ6AGdKWdTaFnH8U3JMvf9ufdPVsStsMnY\npm4b3hrwFs1qN+PkmZO0/rQ1I34cwXuD36NhjYYFvIs5yTL3bU7cTFvftpfOYCgR+gT14ZHQR/j8\n2C2s+/c8aUWbZMRgKBKl3pISkSHA+2iFOE0p9fZF8dWBb9EDE92ByUqpmaUtV17s3KlbUsUh9mQs\nU9ZMoZ5XPaZvnM49He8BLCVVtR6NvBvZvOaeD3+eH0b+QKbK5PXlr9vKuOvXu3jxqhdpWrspmw5v\nop5XPfyrF2r6NUD30yzau4iUsynsOLKD5XctJ/lsMruP7eb6FtfT1b8rbf7XhkoVKrF4zOJc+f2q\n+RFcM5gWdRzflBZ1WhB9PJqxc8fy2+jf8KnqA2iT3Z3t77SleyT0EZbuW8qtbW4t9DVkkTVWKkNl\nmJZUGfP8Vc+zaM8iAq7+kYiIMQwz3X6GMqJUW1IFXDLgIWC7Uqo90BeYLCJOM0MWVUn9tP0nHvjj\nAQ4kH+DZJc/yWNfH+PHGH3n+r+dt/U9ZLalG3o2IORHDrqO7SD6bTN/gvoT6h7IhYQPnM86TkJrA\nzE0zbWOUwmPCizyVT+f6nVl/aD2r9q+ii38XKleozCdDP2HBbQsQEfyr+zOkyRC6N+ieZ+vkxV4v\n5jn1UeUKlXm518ssv2s5VwVclacczes054EuD+TZ31QQfDx9yFAZVPOoRm3P2kUux1B43MSNJ7s/\nyalW/2Nx7m8Zg6HUKG1zX0GWDFBA1myf1YBjSqkLpSxXnkRFQfPm+fvZ2o8LUkrx5oo3efrPp/Gs\n6Em7z9qxNn4tT/d4mpC6ITzQ+QHeX/M+kFtJzds5j+HNh+MmbtSoXIOAGgFsS9rGoj2LCKwRaBuM\nuyRmSY6Bq4WhRZ0WJKQm8OuuX20zf4tIjjFIU6+dytfX5+0FOK7jOJrVbpZn/Ct9Xsk3vqQQEZrW\nalpuTX0i4m2/MGF5Y1izYZyptJ/f1ppRvYayo7SVVEGWDPgYaCUih4DNwGOlLFOenD8PMbEZDP6z\nfp6DYP+K+YumHzW1zf49beM0vt/2PWvGrWHy4Mls/M9GFt620Nan0zeor21Rvywl1aB6A5LSk5i1\nfRbXt7jeVnbXBl1ZG7+WBXsW8ErvVziQcoCdR3eyNn4tfYL65BamALi7udPBrwNfbf4q1/IUWXh5\neOHl4VWk8suaprWblitTn4hEiEh1EakFbAC+EJEpzparKFRwq8ADXe/lSND/iN6TwYaEDcWeestg\nuBSu4DgxGNiolKoPdAA+ERGnvDFjYsCvaQKH0w8Tn5J70milFK9G6PWG3l75NifPnOTlpS/z9Yiv\nqV+tPgCBNQNp5dPKlqd9vfZsTtxMRmaGTUlVcKtAw+oN2XdiXw7l082/Gyv2ryA8JpxhzYZxbbNr\neWbJM7T1bUv1SkWfer2zX2cyVeZlMcj1qe5P2fr4ygk1lFIpwA3A10qprkC5mIbdEfd2GseFlrPo\n+kMjOk/tzO5ju50tkuEyp7T7fgqyZMBdwJsASqm9IrIPaAGsv7iwvJYUKCl27gT/lgeJAw6kHKBx\nrcY54pfuW8qxU8dYOnYpHad2JPp4NNc2u5YOfh0cF4h21/bx9GHP8T02JQXQuFZjujXolsPs1rVB\nVx5d9Cht6rbB18uXES1GMHzWcF7tXbyF+Lo16MbGwxupUrFKscpxBTrX71wq5ZbIkgKOqSAifsAo\n4KXSOEFZUr9afR4N/JLV8xvhNeJ59p7Ym2PmD4OhpCltJRUJNBGRQCABuAUYfVGaOPSX5SoR8QWa\nAQ4nt7NXUiXFmTPw889w4426P6p2sLZOHkw5mCOdUoqJyybycu+XCawZyONdH2fS6klEPxLtqNgc\ndPDrwMbDGzmcdhhfL18ARrUalasfJ6RuCBXcKtjGPQ1sNBDPip5F7o/K4sZWN+Y5w4NBc/FHz8SJ\nE0uq6NeAxcBKpVSkiDRCL69Rbnn1xhvxfwxuGhOcY6Jjg6E0uKSSEpFHgG9V9oJqBUYplSEiWUsG\nZLmgR4nIf3S0mgr8HzBTRLImXHlWKXW8sOcqCosWwYMPQkYG/P03XLgAVTocgGQ4kHwgR9qtSVuJ\nS47jlpBbAO2Se3PIzTalkx8d6nXg731/U6lCJVtflSOTVQW3Ctwacis3tboJgCoVq7B23Fpa+7Qu\n1nW6u7nnO8ODofRQSs0GZtsdxwAjnSdR8aleXc/Icj4pmH1VjZIylC4FaUn5ApEisgGYDixWhegt\nVUotAppfFPa53X4Cul+qzLnnHvjsM+jbF7p2hdhYuDbsIIEEciAlp5JasncJ1zS9xjZfXEX3igX2\naOtQrwNT/51qM/XlxxfXfZHjOGs9JUP5RETeQX+InQYWAW2BJ5RS3zpVsGIybBj8ui2YU96RzhbF\ncJlzSccJpdTLQFNgGnAnEC0ib4hI43wzujgpKXDyJFxzDXh5wdy54OcHpz0O0L1h91zmvvB94UU2\nu3Xw68CBlAMFUlKGy45BluPEMCAWaAI841SJSoBhw2DzcmPuM5Q+BfLus1pOh63tAuAN/Gx9JZZL\ndu+Gpk3BzboDzZpBdDQknj5A9wbdc7Skzl44y6r9q+gb1LdI5/Lz8sO3qq9RUlcmWdaKa4DZSqlk\nZwpTUjRuDLUkmD1HjZIylC6XVFIi8piI/Au8A6wC2iilHgA6UY5t67t2acVkj4h2mOjRsEeOltQ/\nB/+hRZ0WRe7XERE6+HWgXlWjpK5A/hCRnej68peI+ABFW4nSxRgxqA5nzp+3LURpMJQGBWlJ1QJu\nUEoNVkrNtmaOQCmViTZhlEt274bmF3nOXsi8QFJ6Em1925JyNsW2aF94TDgDGzmeFqigDGo0qNzO\nlGAoOkqp54EeQGer7qSTe9aVcsnw6wT3FGPyM5QuBVFSCwGbt501er4rgFIqqrQEK2127YLfqg8l\n+li2N3BCagI+VX3wcPfAv5q/rTWVtU5ScXii+xPlbRCqoQQQkYrAGOBHEfkZuAfIvdJkOaR7d8g8\nHsy66IIrqZNnTjJ7u1mH3tV46a+XXPZjoyBK6n+A/eT8aVZYuWZrXDxbTi1k9o7sCnMg5QANq+tl\nJBrWaMjBlIOcOH2C7Ue2071hd2eJaijf/A9t6vvU2jpyGdQfAHd3aOQdzOJ1BX+5rT6wmmeWlHu/\nkcuOX3b+wr8JRVvnrbQpiJISe5dzy8xXrhdLzMyEPRnh+Hs1sE3iCnpsVIPqDQBoUL0BB1IOMD96\nPv2C+1G5QmVniWso33RRSt2hlFpqbXcBXS6Zq5zQrUUw6/cWXEnFp8QTlxxHytmUUpTKUFgS0xJz\njQ11FQqipGJE5FERqWhtj5HHjBCuztyouSSmJRIfD25Nw3mhl15GI+vPOZhyMLslVb0hB5IPMHfn\n3ByTwBoMhSTDfriGNeNEhhPlKVEGhQZz6NQ+jh4tWPr4VD0r2o4jO0pRKkNhOJdxjhNnTrA/eb+z\nRXFIQZTU/eiO33j0LOZdgftKU6jS4o2VbzBx2UR27lRkBoZzddOrGdZsGL/u+hXQ5j77llT08Wjb\nZK8GQxF5Bvjbmg19GbAUeMrJMpUYresHUz1gH6+9VrD0h1IPUcGtAtuStpWuYGXA1H+n8mnkp84W\no9gkpScB5JrAwFUoyGDeJKXULUqpukopX6XUrUqppLIQrqRJSk9i5qaZzNv2J5Xcq9DIuxEjmo9g\n3s55gNWSqpHdkpoTNYeOfh2p41nHmWIbyjFKqb/Qg+EfBR5Bz75yyqlClSDBNYM5UzmWbxZvJ2JD\n9rCNo6eOMvKnkVzIzLk0XHxqPD0a9mBr4tayFrXE2ZCwgYjYCGeLUWwS0xJxE7fy25ISkcoi8pCI\nfCoi07O2shCuJFFKkZSexJ3t72TGybG0qqy99QY3GUzkoUjm756fy3Ei7VwaI5qPcKbYhssApdRZ\npdQWazuL3Vx+5Z1qlarRK7AXlW4bxYB5LYk7GQfAvJ3z+CXqFxZEL8iRPj4lnsGNB7PtSPlvSSWk\nJZSq2bKslEZieiKtfVqX35YU8A1QDz2/3jL0chuppSlUaZB2Lg033GlzbDxnMlPp01ArKc+Knvx8\n08889edTrItfl8PcBzCihVFShhJHnC1ASbJ4zGLintuO+/Yx/G+1npJwTtQcrml6DZ//+3mOtPGp\nlpK6DMx9CakJ7D62m/MZ5wHYe3wvR08VsHPuEhxOO0zTj5qWWHn5kZSeRLt67Th++jhnL5wt9fMV\nloIoqSZKqVeAdKXUV+jpXbqWrlglz/bYJM4cq0vEfF/GXFjG08Oylc/AxgPZ8sAWFt620KacfDx9\nWDp2KYE1A50lsuHy5bJbzrZSJWin7uCbLV9x4vQJVu1fxYzhM1h7cC2xJ2MBPb1Y8plk2tdrz7mM\nc7a+kOLw8IKHWX8o19JzZUJCWgIe7h7sOb4HgCcWP8HMTTNLpOzDaYc5l3GOb7eU/jzEiWmJ+Hn5\n4eflZ3NscSUKoqTOW78nRSQEqAHULT2RSh6l4LnXkqhXrS4//ghfv9kFn1oeOdJ4uHswpMkQRPRH\nrojQN7hoc/UZDCLyu4j85mD7HajtbPlKg4GtunLmjPDiXy/Sv1F/fKr6cFub2/hyw5eAfqnX86qH\nu5s7IXVD2J60vdjnnLdzHhsTNha7nMKSqTJJSk+iV2AvdhzZQabKZNWBVSU2IDYpPYmalWsyfeN0\nHC06oZSyzYhTXBLTE/Gt6kvDGg1d0g29IEpqqoh4Ay8DvwE7gLdLVaoS5scfYd+RJDo0K1e61VC+\neReY7GB7FxhakAJEZIiI7BSR3SLynIP45iKyWkTOiMiTF8XFishmEdkoIuuKfTUFoGuoUOvAWD77\n9zNGttTTeo7rOM7WGohPice/uj8AIT4hbE0qnvPEsVPHiE/V464Kg1KK1QdWF/vc1Tyq0d63PTuO\n7CDqSBTHTx8nNjm2WOVmcST9CEOaDCH9fLrDluJ7a96j8xedycjUoxki4yP5ecfPRTpXYnoidavW\npWH1hi7pPJGvkhIRNyBFKXVCKbVcKdXI8vL7PL98rsaHH8L1Y3RLymAoC5RSy/LbLpXfqnsfo/uC\nWwOjRaTFRcmOoT0GJzkoIhMIU0p1UEqFFvNyCkSXLnAk/HbqeNaxDdsIqRtC6rlUElITiE+Nx7+a\nvy28uB5+WUqusC/WfSf3cdX0q4rV35OQloBfNT9a+bRix9EdrNy/ko5+HW2mzeKSlJ6Eb1Vf7m5/\nN9M2TssRdyD5AG+seAPQfX8ZmRmM+30cU/6ZUqRzJaYl4uvlS0CNgFzOE64weXC+SsqaXeLZMpKl\nVDh/HjZvhhr19deCwVBOCAWilVJx1sS0s7hoYlql1FGl1L/o5XMuRijgUjwlhZ8fVMsMYMWIQ9Ss\nXFMLIUKofyhr49dyKPUQ9avVB6BvcF/m7ZrHpFWTbK2BwrL58GZa1GlR6JbUtqRtKBRL9i4p0nlB\nj/fy8/Kjdd3WbE/azor9K7i97e3Enox1aJ4rLEdOHcHH04ex7cby0/afcrjyP7boMR4OfZhJAyfx\n+vLX+WrzV1R0q8jmxM02J46L+W3Xb/xz4B+HcTZzX/Wc5r6MzAyCPwjmgzUfFPt6ikNBHuJwEXla\nRBqKSK2srdQlKyG2boVGjeDk+SSjpAzlCX/A/rP2oBVWUBSwREQiReTeEpUsH7p0gU3/VswR1tW/\nK+vi12lzn9WSauXTish7I/l99+/c/PPNRXqxb0ncwrXNrrW5vReUbUnbqONZh0V7FxX6nFkkpOqW\nVIs6LYg+Hs2yuGVc3eRqqlSowpFTRwpUxoq4FWSqTIdxSen6fdWwRkMaVG9AZLxeAXnV/lVsTtzM\n81c9z9VNrqaSeyUeXvAwHw/9mKCaQXl6TX649kMm/zPZYVxWS6phjYbsT8lulUYdjaJqxapM/mdy\nmThw5EVB5uC72fp9yC5MAY1KXpyS5e2Vb5O0oTuhob1JSk+iewMzSazB+YhIBaWUo9ZPSdJTKZVg\nrV+1RESilFIrHSWcMGGCbT8sLIywsLAinzQ0FNatg1tuyQ7r6t+Vd/95F9+qvrSr184WHlQziPCx\n4fSY1oNPIz/lodCHHJSYN1uStvDuwHf5YO0HZGRm4O7mnmfa5XHL6RXQCxFhW9I2Huz8IJ//+zmZ\nKhM3KXyDMyEtAT8vPzwreuLn5UfauTSa1W5GUM0gYk/GXvKDWCnFNd9fQ/jYcEL9c1tjj5w6gk9V\nHwCGNBnC4r2L6d6wO99t/Y5xHcbZ5hJ9Z+A7zN89n24NutGlfhciD0XSwa9DjrIuZF5gXfw6RIT0\nc+lU9aiaI+7EmRPU8ayjzX12LanI+Ej6BPXhpV4v0e+rfriLO6PbjC7UfYqIiCAiIqJQeS7mkkpK\nKRVcrDM4kTlRczi/7zT3h/bmx3TTkjKUHSKyUil1lbX/jVLqdrvodejZ0PMjHgiwO25ghRUIpVSC\n9XtEROaizYeXVFLFJTQUxo/PGdbFvwvrD62nrW9bm7kvCw93D74f+T09p/ekQfUGnLlwhpC6IbSu\n2zpHupSzKVRyr0SlCpUAbYracWSHbUaYQ6mHbLPFXMyR9CP0mdmHHQ/uoKVPS7Yf2c5T3Z/ih20/\nsPnw5lwv9YKQkJpAk1pNAN0q9HD3QEQI9tbrazlSPPYcO32M1HOprDm4xmHapPQkfDy1khrceDAv\nLX2Jl3u/zJyoOay+O9vpo19wP/oF9wMg1D+UyPhI7uuUc9a6bUnb8K/uT0CNABZEL+Cm1jdly3Hq\nGN6VvangViGX40TkoUg6+3WmlU8rwseGM/jbwaSeS81Vfn5c/NEzceLEAufNoiAzTox1tBX6TGWM\nUoodR3YQk7qd0NDs5rPBUEZUtdtvfVFcQQbzRgJNRCRQRDyAW9DetXlhK1NEPEXEy9qvCgwCymT0\nbKdOsGULHMyeIYk6nnXw8fRhXfw6m7nPnma1m/HR1R/xfyv+jw/XfcgLf72QI3570nZCPg3h6T+f\ntoVFH4+mnlc9qlWqRkCNgHydJ5bHLQfgr31/cT7jPLuP7aalT0tbC6UoZDlOgFYiN7S8AYCgGkEF\ncp6IOaHn6F4bv9Zh/JH0I7b31VUBV7HjyA5+ifqFhtUb0rhWY4d5utTvwrpDuR05Vx9YTY8GPbix\n5Y05liYCqz/KyxeAWlVqcS7jHKln9VwNkYci6eKvJ+wPqRtCxB0RvPjXizbZy4qCtHO72G29gAnA\ndaUoU4mwP3k/py+cJrXKDkJCjJIylDn5dbJcsgNGKZUBPAz8CWwHZimlokTkPyJyH4CI+IrIAeAJ\n4CUR2W8pJ19gpYhsBNYAvyul/izm9RSIGjXgySfhP//R4xOzCPUP5VzGOZsL+sXcEnILkfdG8suo\nX1ixf4XNmWLNwTX0/aovT3Z/ku+2fkdiWiKg+6OyVroOrBGYr/NERGwEHep1IDwmnD3H99CgegM8\nK3oypMkQFu5ZWKTrzDL3ATzS9RHGtB0DYDP32ZN8JpnZ22fz8IKHbQOYY07E0NqnNWsOrnFYvr25\nr1KFSvQO7M2Ti5/k5tY3O0wP0Na3LXuO7+HU+ZxTQ64+sJoeDXtwfcvrWbx3cY74xLRshzIRIbBm\nILuO7eLshbNsT9pOh3rZrcymtZtyV/u7ck2qe+r8qVzTX5UkBZlg9hG77V60mcKr1CQqIXYc2UHr\naj3AO4YMOcOJMyeo7XlZjqE0uCY1ReR6ERlp7d9gbSPRA+IviVJqkVKquVKqqVLqLSvsc6XUVGs/\nUSnVUClVUylVSykVoJRKU0rtU0q1t9zP22TlLSteeAHi4+Gbb7LDuvp3pXql6nh55P/q8PXyxc/L\nj82JmwF4ffnrvNH/DR7v9jijQ0bz/pr3Aa2k2vnq/q2AGgH5Ok8si1vGa31fIyI2gk2HNxFSNwSA\nvkF92Xt8Lyv3O7SC5kuW48TFBNUMyjFWakviFlp92ooZm2YQERvBslg9+iDmRAxXN7maY6eOcSQ9\np6PF2QtnOX3+NDUqZT8mQ5oMIT41nlGtR+UpU6UKlWjt0zrX4OYsJVXHsw5d6ndh/u75trgsz74s\nbmtzGx+v+5gtiVtoUqtJjv4rgAe7PMjMTTNJP5duC/vi3y8YNXtUkb00L0VRXFTTAZfvp9pxZAfV\n0ztSg4asObiGmpVrUsGtXK/VaChfLENbHIZZ+9da2zBguRPlKnU8PGDGDHj4YahdG264QZusgmsW\n7LURFhTG3/v+5sTpE6yIW2FrPTzT8xmmbpjKhIgJfLb+M/oG6RlhAmsE5mnuO3rqKHHJcQxpMoSA\nGgF8tfkrQny0kqpSsQrvDnqXhxY8lMPFOyMzg/iUvLv/lFI5WlL2ZPVJgW4FDvxmIO8Nfo8Fty1g\nZMuRbEncAsC+E/toUqsJXRt0zWXyy2pFZc1+A3B9i+t5uvvTl5ymLct5IouE1ASSzybTvE5zAO5o\nd0h6AvsAACAASURBVAfTN2XPD56YllNJPRz6MH/s/oPZO2bTpX7utTmDvYPp0bAH32/9HtD36v21\n7+Mmbmw/UvwZRBxRkD4p++ld/gB2AXMvlc/Z7Diyg/TYVjSu3oq/9/2d448wGEobpdRd+W3Olq+0\n6dABTpyAnTth1Sqofa4Tq+8p2CwPfYP6EhEXwa+7fmVAowFUq1QN0K2UO9rdwe5ju1kzbg19gvoA\nEFgzb3Pf8rjl9GzYkwpuFRjQaACL9y62taQAbm59M7Wq1OKz9Z/Zwqb+O5Vu07rl2TJIPptMRbeK\nuVoZkG16TDuXxk2zb+LLa7+0tX7a+LaxDUCOORlDI+9GdPPvlsvkZ+80kYV/dX8mDXI0ZjsnXRt0\nzTGbxj8H/6F7g+42D8YbW91IZHykzSRp3ycFULNyTe7rdB+T/5ls64+6mEdCH2HKmikcP32cuTvn\n4uflxw0tb8jTdFlcCtKSsp/e5U2gt1Lq+VKRpgTZcWQHsetb0a1xKyLiIkx/lMFlEJFLefZdFri7\ng48PDBoEixfrFQcKQp+gPqyIW8EP237gplY35YibMngK34/83uZZB+RynEg7l8YPW38g7mQcEbER\nhAWFATCgkV75wF5JiQgfXf0RE5dN5Pjp4yil+CTyE06fP82iPY7HUeVl6gOo6lGVah7VeHLxk/QO\n7M21za+1xbWpa6ekTmgl1bVB11wvd3unicLSO7A3K/avsI07Wx63nB4Ne9jiq1Sswpi2Y2zzKV5s\n7gN4otsTVHKvRFd/x/OID2g0gCGNh9BpaicmREzgqe5P0dW/K2sPOnYCKS4FUVL7gbXWdC6rgGMi\nElQq0pQQSim2Je3AM10rqTUH1xglZXAlHnC2AGXJ4MGwqBDjZutWrYt/dX+Wxy0v0KrYWa0XpRRv\nrHiDoPeDmL5pOh2ndmT6xuk2JdU7sDct6rSgae2mOfKH1A3hhhY38OaKN1mxfwUXMi/wZv83+WLD\nFw7Pl5epL4tg72C+2/odbw/IOcVpk1pNOJx2mBOnT3Ao9RABNQLo6t+VyEOROcyNSelJNqeJwhJY\nI5CKbhVtM7Mv3LOQIU2G5EhzX6f7mL5xOr9E/UJ4TDjB3jnNsD5VfYh5LCZP13wR4b0h7zF50GSC\nagYxosUIujXoxpp4rWwPpx3mx20/Fkl+RxSkk2Y2evn4LDKsMMdtQRfgUOoh3DKq0KdLbVr7tOZc\nxjmjpAwug+WAdMUwaJDunzp/HipWvHR6gLDAMFrUaWEz9eVHjco1cBd3xv02jq1JW1k7bi2NazUm\n5WwKf+79k871OwPg5eFF1ENRDsuYEDaBkP+FEHkokge7PMjoNqN5Lvy5HFM5ZXEo9VCeLSmA1j6t\nGd58uG3Znyzc3dxpWaclC6IX4OflR0X3itT2rE2ofyhT/pnCsz31DHRHTh2hrmfR3lciQu/A3iyP\nW46buNmWRrGnlU8rmtVuxoSICXx57Zf0CeyTq5x6XvUuea4bWt5gc71v49uGuJNxJJ9J5vVlr/P5\n/7d35/FVVdcCx3+LScaEoEwygwyCgCAzWFCsiDjwVBRUnCjCE6lTraj9VLDWDqLWOlThWVqtsxWI\noigoOCBUCwEqg4CEeShaMECRIVnvj31CLiEhd8y55971/Xzux3tOztlZidmsu/fZw+Ln6FCvwzGt\n1miFk6QqqeqhwgNVPeTN20haK3atoMZ/29OvH7Q9pS2CWJIy5UpEmgF7VPV77/gcYCiwEXgqtE6l\nunr1oFUrWLQIzj47vHsmnTOp1HXoStI0synLdi5j7nVzj64bmHFSBle0vyKs+xvWasit3W9l8sLJ\nzBw+k5pVajKs/TCm5Uzj/h/df8y1n2/+nJa1S19wZ8rFU0odpNWxfkdmfj2TlllF90+9eCrdp3Zn\nSOshdKjXIaaWFLgW4yebPuHAkQMMbj24xBU13rvmPapUrHLCVToiUalCJbo27Er219m88tUrTOg3\ngdtn386ckXOOGQASjXC6+3aJyNF5USJyKZD47SJjsHLXSg5s7EC/fq4fvEVWC0tSpry9jjehV0TO\nxPU+bAI6A8+c4L6UFGmX3ynVTzlha6W4KRdPYc7IOUcTVDQm9JvAnJFzyKzqhn6P7TaWZxc/e0yy\nXLRlEdNXT+eO3neUWs6JRhF3rNeR99a9d0ySal67Ob8+99dcP+N6CrSAXft3HTdwIhKFLan31r3H\n4NMGl3hNtcrV4pagCvVq3IvbZt/GyE4jeaD/A2zft50Zq2fEXG44SWoscJ83UXATcA8wJubvnCBz\nvpnDEwuf4mBuDzp48/zPbno2bU5u429gJt1UU9Vt3vtrgT+r6qPAjbglitLKhRfCyy/DxsjWgg1b\nr8a9yKqWFVMZ1SpXO2aQQZeGXTitzmlHV2k4eOQgo7JH8cQFT3BK9VOi+h6d6ndi36F9xyQpgNFd\nR1OgBby/7n3X3RfDh+q2J7dl/6H9fJT7ET9u+eOoy4lUz0Y92XdoH3f3vZvKFSvzzIXPMPrt0YzO\nHh3TZpDhTOb9RlV7Ae2B9qraR1XXRf0dE+jxhY8zdtZYhmc9Rv+sa6nofVD4y9C/HH14akw5Ce3j\nOBf4EI5uf5N2+vaF8eOhZ0+YN8/vaMJ3V++7eHThoxRoAePeHUebk9scN+IwEh3rdQQ4LkmJCON7\njOepL5+Kubuv8LlUt1O7xZy4IzG49WDeHvH20Wdx/Zv3Z834NTSo2eC4VSoiIWUtkS8iDwO/V9U9\n3nEWcJeq/iLq7xoFEdGyYj3/xfO5redtzJsyhKwsuP/+E15uTKlEBFWNujNdRJ4AGgLbcZN626jq\nYRFpiFumqFucQo1JOPUqnubOhWuugYcegtEBGD5SoAW0f7o9TTObsv/wft6/9v0yV80oS/3J9cke\nnk3PxscO8T5w+ABN/9CUQ/mHWHzz4mOG2UfqnTXvsP/Qfq46o/RllPwQTb0Kp7tvcGGCAlDV3YS5\n/XV5UlVyduTQpWEXliyBbknxT4BJY7cDbwEbgH7exoUADYC0/fh03nnw2WfwyCPwi3L9mBudClKB\n+86+j/2H9/Pu1e/GnKAA3hnxztERh6GqVa7GTWfeRN7BvJieSQFc1OaipEtQ0QqnJbUc6K6qB73j\nasA/VbX4ys4JVdYnvi15Wzhryllsv3MHdesKK1ZAg7JHURpTolhbUkFR3i2pQt99B336wM9/DqNG\nlfu3T1ob9mygx9Qe7PzZzphHxSWjaOpVOEPQXwI+FJFpuH72G4C/Rh5eYuVsz6FLgy5s2yZUqmQJ\nyiQvEZmiquFvypOCTj4ZsrPdkPS2baFfP78jSg7Nazdn651bUzJBRSucgRO/Ax4CTgfaAu8DJ17l\nMISIXCAiq0VkjYjcU8LXfyYiOSKyRET+JSJHRCTicaQ5O1ySWrYMOncu+3pjfPSc3wEkg7Zt4dln\n3URfHxpzSatyxTBnPKeJcFdB34nbA2cYbqRSydO2ixGRCsBTwCDcxm8jRKRd6DWqOtnbUqArcC8w\nP/QZWLiWbF9Cl4ZdWLrUkpRJbqq62O8YksXQoZCXB4vtN2JKUWp3n4i0AUZ4r2+B13DPsM6JoPwe\nwFpV3eiV+SpwKbC6lOtHAK9EUP5ROTtymHz+ZN5cBpck/ZaMJtWJyIl20UVV7a8UqFDBPZP6v/+z\nwU6mZCd6JrUa+BS4qHBelIiUPs26ZI2AzSHHWyhlIqM3IOMCYFyE34Pv/vsde37YQ8uslixbBr/8\nZaQlGBN3vXF/+68A/yC8LePT0g03QMeOMHky1Ez67VRNeTtRd99luDke80RkqogMJLEV7WLgs2i6\n+pbuWErn+p058N8KbN7s+rqN8VkD4D7gDOAJ4MfAt95uAh/7GlmSadTIDZx44w2/IzHJqNSWlKrO\nAGaISA1cF93tQD0R+RMwXVU/CKP8rUDTkOPG3rmSDKeMrr6JEycefT9gwAAGDBgAFA2a+OoraNcu\n/JWWjSk0f/585s+fH7fyVDUfmA3MFpGTcF3Z80Vkkqo+FbdvlCJGjXItqRtTfjtIE6ky50kdc7Fb\nbWIYcJWqDgzj+oq4nXwH4lplXwAjVHVVsesygfVAY1U9UEpZpc7nGPP2GDo36EylnFtYuNBtXW1M\nLOIxT8pLTkNwCao5kI1bw6/0vcnLmV/zpIo7dAgaN4aFC92K6SY1JWrFiaNUdbeqTgknQXnX5wO3\nAh8AK4BXVXWViIwRkdB5IkOB90tLUGXZ8P0GmtduzpdfQte02PPUJDsReQFYCHQFJqlqd1X9VTIl\nqGRSpQoMHw5/+5vfkZhkE1FLyk8n+sTX5sk2zBw+kyE9Tyc7G86IfZ8tk+bisHZfAbDfOwz9wxVA\nVTUjlvjiJVlaUuCGoV95JaxbBzaXNTUlvCWVjAq0gE3fb4I9zfnvfzm6PYcxflLVCqpay3tlhLxq\nJUuCSjZdu0LVqvD5535HYpJJ4JPUtr3byKqWxYKPqzFwoH0CMyaoRNzAiV/9CvLz/Y7GJIvAJ6nc\n3bm0qN2CuXNhYFhPyowxyeq22+DwYbfwrDGQAklqw54NNMtszkcfWZIyJugqV4Y334RZs+DFF/2O\nxiSDwCep3D251DzcgsxMaBb2srfGmGSVlQXPPw+TJlm3n0mRJLV3cwtrRRmTQvr0gTp1XIvKpLfg\nJ6nduXz3TQt6lLgioDHBFcY2N21F5HMR+UFE7ozk3mQnArffDn/4g9+RGL8FP0ntyWXrVy3o2NHv\nSIyJn3C2uQG+A8YDj0Rxb9K74gpYswaWLvU7EuOnQCepw/mH2b53O+uXNrH5USbVHN3mRlUPA4Xb\n3Bylqt96e1MdifTeIKhSBe69F0aMgM2bYetWGDDAFqJNN+FsH5+0Nudt5pSqDalxamWqV/c7GmPi\nKuxtbuJ8b1IZNw5++AH69nWDKFq3hrffhmHD/I7MlJdAJ6nc3bnU1hac3snvSIwxiXLXXW7kbmYm\nNG0K55/vtpu3ifvpIdhJak8ulffb8yiTkiLZ5iame0vbAieZXHGF+68qHDwIGzZAixa+hmTCEI8t\ncAK9wOy4WeOYP70VDw6+k8sv9ykwk5LisVVHjN8/rG1uvGsfAPap6qNR3Js0C8yG66qrYPBgt6Mv\nuMR1zz0wdiy0bOlraKYMabfA7ILNC9i9rK+1pEzKCWebGxGpLyKbgTuA+0Vkk4jULO1ef36S+Ovf\nHz4O2dt42jS3YaLtI5eaAtuSyjuYx6mPnkrBb/7D3j1VqFjRx+BMyvG7JVVegtiSWrECLr4Y1q+H\njRuhWzf47W/hkUdg1Sp7VpXM0qoltWjLIlrXPIsO7SxBGZNO2reHvXvh8cdh0CA3sOKmm9wowK++\n8js6E2+BHTixYNMCGhzqS0Pr6jMmrYjAZZfB7Nnw2GPu+ZSIG1zx+utY93+KCWxLasHmBVT7ti9t\n2vgdiTGmvD33HLz/Plx4YVH33rBhbqJvwHovTRkCmaSOFBzhi61fcCS3t43mMcYA0KOHG57+2Wd+\nR2LiKZBJatmOZTTJbMKWtXVsroQxBnAtqkcfhVGjYP9+v6Mx8RLIJDV/w3zObno2ubk2L8IYU+Sy\ny1yL6t57/Y7ExEsgk9TMr2dyzqkXU1Dg9pwxxphCTz7p1vcbMcJG+6WCwCWpf+//N8t3LqfJkYG0\naGFzIowxx8rKguXL4cwz3arpMa7KY3wWuCSV/XU2g04bxLZNVa2rzxhTolq13FJJL74I118Pe/b4\nHZGJVuCS1IzVMxjadijr19sCk8aYExs8GC66CG691e9ITLQClaT2HtzLJxs/4cLWF9qgCWNMWB55\nBObNc8spmeAJVJKat2EePRv3JLNqprWkjDFhqV4drr4aXn7ZHau6rT5MMAQqSe3Yt4Pmmc0BrCVl\njAnbNde4JKXqVqpo1cotRmuSX6CSVN7BPDJOyiA/HzZtgubN/Y7IGBMEnTtDtWpuNYq774aBA+GX\nv/Q7KhOOQCapbdvg5JOhalW/IzLGBIGI6/K77jo3RH36dFiwAHJy/I7MlCWQScqeRxljIjVihNt/\navJkqFHDrUpx//1+R2XKEqgktffg3qNJyp5HGWMi0aoVbN7slk0CuPlmWLfOrU4B7nlVXp5/8ZmS\nBSpJ5R1yLalVq6BdO7+jMcYETaNGRe9POgmeeQbGj3fJacwY9+F361b/4jPHC1aSOphHrZNqsWIF\ndOjgdzTGmKA77zzo29f9e7J+PYweDSNHQn6+35GZQoFLUhknZbBypdtC2hhjYvXYY277+Vmz4KGH\noKAAfv97v6MyhUQDso2liGj7p9sz7cLXGND+DPbuhYoV/Y7KpCoRQVVTfvliEdGg/BtQXnJzoVs3\n9/yqenW/o0kt0dSrwLWk/r05gzZtLEEZYxKjRQvo0wdee83vSAyUQ5ISkQtEZLWIrBGRe0q5ZoCI\n5IjIVyIyr7Sy8g7msWlthj2PMsYk1Nix8OyzfkdhIMFJSkQqAE8Bg4AOwAgRaVfsmkzgaeAiVT0D\nGFZaefsO7WP96lqWpIwxCXXBBbBjByxc6Bao/fWv/Y4ofSW6JdUDWKuqG1X1MPAqcGmxa64G/q6q\nWwFU9dvSCqtWqRqrV1a0QRPGmISqWNGN9PvRj9xSSs8/71apiERBgZt7ZWKT6CTVCNgccrzFOxeq\nDVBHROaJyJciMrK0wmz4uTGmvNxxB3z+Ocyc6RanHTvWDaYI1//8jxsxaGJTye8AcDF0Bc4FagAL\nRWShqq4rfuGhuYf4dstEXnwRzj13AAMGDCjPOE0Kmz9/PvNtn3ETokYN6N7dve/VC265BSZMgJde\nKvveI0fgo4/cB+qLLkpsnKku0UlqK9A05Lixdy7UFuBbVf0B+EFEPgE6A8clqfqDWtJk+0QefDBR\n4Zp0NWDAsR96Jk2a5F8wJindcgu0bg3797sEdiLLlrnrbAHb2CW6u+9L4DQRaSYiVYDhQHaxa2YC\n/USkoohUB3oCJe70UuFQBm3bJjReY4wpUd260Ls3ZBf/F8zzpz/BBx+49598AkOGwOLF9lwqVglN\nUqqaD9wKfACsAF5V1VUiMkZEbvauWQ28DywHFgFTVHVlSeXJ4QwaNEhkxMYkjzCnb/xRRNaKyFIR\n6RJyfoOILPOmdnxRflGntquvhldece9XrYI1a9z7I0dg0iT41a/c8aefulXXVWHbNn9iTRUJfyal\nqrOBtsXOPVfseDIwuczCDmZQt25cwzMmKYVM3xgIbAO+FJGZ3oe6wmsGA61UtbWI9AT+BPTyvlwA\nDFDV3eUcekobOhRuvRXmzoUrr3QLXS9Y4FpQTZrAN9/AihUuST3xBHTt6rr8GhUfLmbCFqgVJwoO\nWJIyaSOc6RuXAi8AqOo/gEwRqe99TQhY/Q6CWrXg/PPdYIipU2H3bpewXnjBrf93001w111Qs6ZL\nWl26wJIlfkcdbMkwui9sh/fVsiRl0kVJ0zd6lHHNVu/cTkCBOSKSj+tCn5rAWNPKAw+44egDB8IP\nP7jNE9euddt+fP89PPwwXHutu7Zr16LuQROdQCWpQ3utJWVMmPqq6nYRqYtLVqtU9bOSLpw4ceLR\n98VHOZrjnXGGewEMHw4PPui2/KhTx70uucStWAGuJXX33f7F6rd4TO0I1Cro9YY8zceTb7END03C\n+b0Kuoj0Aiaq6gXe8QRAVfV3Idc8C8xT1de849VAf1XdWaysB4C9qvpYCd/HVkGP0eLFrnuvcORx\nfn7RAtgFBVC7NqxcCZUqQf36ICm/tn7pUn4V9H3fWUvKpI1wpm9kA9fB0aS2R1V3ikh1Eanpna8B\nnA98VX6hp5ezzuKYqTGhOzRUqOBWVD/tNGjWDEIbFYXdg+bEApWkfvg+g6wsv6MwJvHCnL7xLpAr\nIuuA54BbvNvrA5+JSA5uWsfbqvpBuf8QBoB334UDB+C++9x7cMPSx41z58yJBaq7r/aZ89idM8Dv\nUEwa8Lu7r7xYd1/5WbQIbr4Zli93IwPfeguWLoUZM6BnT7+jKx8p391Xp0aG3yEYY0xUuneHrVvd\nKzsbrrsOHn3UJa68vBPfqwobNhQdHzjgkl06CFSSOiXDkpQxJpgqVnSjAGfMgI8/diMAR4yAfv3c\nc60Tzad65x03nP3wYXc8darbRmTXrvKJ3U+BSlL1Mmr5HYIxxkRt0CC3dNJZZ0FWlhvp9/TT7tx5\n50H16tC5M+zZc+x9b73lzn30kTv++9+hZUu4//7y/xnKW6CSVMM61pIyxgTXoEGwcydcfPGx54cP\nh+++cy2jbt04ZqeHI0fg7bdh/Hh48013//Ll8N57rtsw1Ve0CFSSalC3qt8hGGNM1Bo1gquugssu\nO/5rIm4LkN/8xi2z9PXX7vwnn0CLFm4TxhkzXKIaPNjNuZo40a2AkcoClaTq1U35wVbGmBT36qvQ\nvHnpX69Xz22uOH68W3Zp+nS3y2/z5u41aRJcfrm7dsQI93xr795jy/jsM5gzJ0E/QDkLVJKyibzG\nmHTw05+6lSq6doU33nBJCmDYMNi3r2jZpcxM6NvXdf2Fuu8+uPPOor2sZsyAKVOKBl6AS4DvvHN8\ngks2gVq7z5KUMSYdVKkCr73mXtnZcPrp7vwNN7huvtCdgYcOdUnoyivd8bp1sHq1S2CLFkGnTm5B\n3NatYfJk6NXLDXn/9FO3bNMTT7jh8MkqUJN5ly9XOnb0OxKTDmwyrwmK7duhfXs3oKJKFfjFL9zW\n9aee6tYMbN8evvjCtcg+/hhyc91ag716wcsvu/sff7x8Yo2mXllLyhhjAqxhQ9fSmjfPDWN/4QXX\njdewIbRp45Zi+vBDd23//u5VqEsXmDXLn7jDFagkdfLJfkdgjDHJ5/LL3Y7BnTq5D/OdOrnzgwa5\n51KFW4sU16WLW5qpoMAthlu4cWO/fuUXe1kC1d0XlFhN8Fl3nwmS/Hw3VH3lSpd4+vRx5/Py3EoX\noc+wimvSxK3OXr26W629dm3XPZiILe9Tfu0+Y4wxx6tYEc45x62sXpigADIyTpygwCW1nBzXLXjR\nRa5FdvnlcOhQyde/9tqxq2IcOACvv+4SZaj16+Gll6L/mQpZkjLGmDTWtatLUrNmuSQ1YYIbgPHW\nW0XXzJoFf/6zGxo/YYJrZf3O235z4kQYM8atJbh2bdE9U6fCww/HHp8lKWOMSWNdurih6h9+6OZf\nibjW1HPPua9//jn85CduyLqq6wqcMsW93n0Xpk1z3YxXXOEGbhw54u6bPt0NhY91HpY9kzKmBPZM\nyqSLTZvcrsG9e7uEBK6rr2lT96xq3Di3ssVPfnLsfRMmwCOPwLPPwujR7lzv3nDPPdCunUtYjRq5\na370I/f1lB+CbowxJr6aNIE6dWDIkKJzVarAjTfCqFFu/tX11x9/34QJ7nnXqFFF58aNg2eecc/H\nhg51rbJ//rMoSUXDWlLGlMBaUiadPPCAW3WiVauic+vXu9F+f/0rjBwZXjkHD7oWWNWq7hnWli0w\neza88or7ejT1ypKUMSWwJGWMG9bet68bPRiue+91z7N27oQ1a1yLqnBAhSUpY+LEkpQx0dm1CxYu\nhEsuccPSa9d2z73cJo82T8oYY4yP6tZ1CQpcC6xLF1i8OPryLEkZY4xJmG7d3OCJaFmSMsYYkzDd\nu8PGjdHfb8+kjCmBPZMyJj5U3VB0sGdSxhhjkozE+FHPkpQxxpikZUnKGGNM0rIkZYwxJmlZkjLG\nGJO0Ep6kROQCEVktImtE5J4Svt5fRPaIyBLv9YtExwQwf/78pCwr3uVZbMFVVt3xrvmjiKwVkaUi\ncmYk9yZCuvx9WGzlJ6FJSkQqAE8Bg4AOwAgRaVfCpZ+oalfv9VAiYyqULn8UFlswhVN3RGQw0EpV\nWwNjgGfDvTdR0uXvw2IrP4luSfUA1qrqRlU9DLwKXFrCdSk/H8WYCIVTdy4FXgBQ1X8AmSJSP8x7\njQmERCepRsDmkOMt3rnienvdFbNEpH2CYzImCMKpO6VdE269Myb5qWrCXsDlwJSQ42uBPxa7piZQ\n3Xs/GFhTSllqL3uV5yuRdSNOdedtoE/I8Vygazj3Wr2yl1+vSOtConfm3Qo0DTlu7J07SlX3hbx/\nT0SeEZE6qvqfYtdZl6BJJ2XWHe+4SQnXVAnjXsDqlUl+ie7u+xI4TUSaiUgVYDiQHXqB14de+L4H\nbj3B/2BMeiuz7njH1wGISC9gj6ruDPNeYwIhoS0pVc0XkVuBD3AJ8XlVXSUiY9yXdQpwhYj8L3AY\nOABclciYjAmCcOqOqr4rIheKyDpgP3Djie716UcxJiaBWQXdGGNM+gnEihOxTEwUkedFZKeILA85\nlyUiH4jI1yLyvohkRlBeYxH5SERWiMi/ROSn0ZYpIieJyD9EJMcr64E4xFfBmxSdHYeyNojIMi++\nL+JQXqaIvCEiq7zfX88of29tvJiWeP/9XkR+GmNsd4jIVyKyXEReEpEqsZSX7GKd7BvPehXPOuXd\nlzb1Kl51yisrrvUqbnXKzxFMYY5yqgCsA5oBlYGlQLsI7u8HnAksDzn3O+Dn3vt7gN9GUF4D4Ewt\nGpn4NdAu2jIpGtlYEViEm+MSS3x3AH8DsuPws64Hsoqdi6W8vwA3eu8rAZmxlBfy97ENN4Ag2v8H\np3o/axXv+DXg+lhjS9ZXrHXKKyNu9Sredcq7Pi3qVSLqVMjfSNT1Kp51yvcKE8YP2wt4L+R4AnBP\nhGU0K1aZVgP1vfcNgNUxxDcDOC/WMoHqwD+B7tGWhRvFNQcYEFKZoo4LyAVOLnYu2tgygG9KOB/r\n7+184NMYYzsV2AhkeRU9Ox7/T5P1FY865d2XkHoVrzrl3Zey9SpRdcq7L6Z6Fc86FYTuvkRMTKyn\nbhQUqroDqBdNISLSHPdpchHuFx9xmV43Qg6wA5ijql9GWxbwOHA3bj5CoWjLwitnjoh8KSI/ibG8\nFsC3IjLN606YIiLVY4wP3ECbl2OJTVW3AY8Cm3BDtb9X1blxiC1ZJWqyb8z1Kh51yisnHepVeOIA\n+wAABDBJREFUouoUxFiv4lmngpCkykPEo0dEpCbwJnCburlexcsIq0xVLVDVLrhPaz1EpEM0ZYnI\nEGCnqi7lxMtMRfKz9lXVrsCFwDgROTua2DyVcBNNn/bK3I/7BB9teYhIZeAS4I1S7g2rLBGpjVs2\nqBnuE2ANEbkmltgMEOHvK151CtKmXsW9TkF86lU861QQklQ4kxojtVO8+Vki0gD4dyQ3i0glXGV6\nUVVnxqNMVc0D5gMXRFlWX+ASEVkPvAKcKyIvAjuijUtVt3v/3YXrgukRZWzgPq1vVtV/esd/x1Ww\nWH5vg4HFqvqtdxxtWecB61X1P6qaD0wH+sQYWzJLRJ2CGH5fiahTkPL1KhF1CuJTr+JWp4KQpOIx\nMVE49lNQNnCD9/56YGbxG8rwZ2Clqj4RS5kickrh6BYRqQb8GFgVTVmqep+qNlXVlrjf0UeqOhK3\ndE5EZXnxVPc+2SIiNXB91P+KJjYvvp3AZhFp450aCKyItjzPCNw/HIWiLWsT0EtEqoqIeLGtjDG2\nZBavyb7xrFdxqVOQPvUqQXUK4lOv4lenIn2g5scL9ynoa2AtMCHCe1/GjVI56P3ibsQ9zJvrlfkB\nUDuC8voC+bgRUTnAEi++OpGWCXT07l8KLAfu985HXFaxcvtT9IA3qrJw/d2FP+O/Cn/vscQGdMb9\nA7kUeAs3Eina+KoDu4BaIediie0B3D9ky4G/4ka9xfT/IZlfsdQp7/641at41imvvLSpV/GsU155\ncatX8apTNpnXGGNM0gpCd58xxpg0ZUnKGGNM0rIkZYwxJmlZkjLGGJO0LEkZY4xJWpakjDHGJC1L\nUgEhIvkhS+gvEZGfx7HsZiLyr3iVZ0xQWL1KfgndmdfE1X5163Mlik2YM+nI6lWSs5ZUcJS4uKWI\n5IrI77yNxRaJSEvvfDMR+VBElorIHBFp7J2vJyJveedzRKSXV1QlbxXlr0RktoicVE4/lzF+snqV\n5CxJBUe1Yt0Sw0K+tltVOwFPA4Vrnz0JTFPVM3FL2Dzpnf8jMN873xW31hdAa+BJVT0D+B64PME/\njzHJwOpVkrNlkQJCRPJUNaOE87nAOaq6wVtJeruq1hWRXUADVc33zm9T1Xoi8m+gkaoeDimjGfCB\nqrb1jn8OVFLVh8vlhzPGJ1avkp+1pFKDlvI+EgdD3udjzyuNsXqVBCxJBceJNly7yvvvcGCh934B\nbsl9gGuBT733c4Fb4OjupYWfIk9UvjGpyupVkrOsHhxVRWQJ7o9egdmqep/3tSwRWQb8QFEF+ikw\nTUR+hlt6/0bv/O3AFBEZBRwB/he3xbb1+5p0ZPUqydkzqYDz+s7PUtX/+B2LManC6lXysO6+4LNP\nGcbEn9WrJGEtKWOMMUnLWlLGGGOSliUpY4wxScuSlDHGmKRlScoYY0zSsiRljDEmaf0/x7f1amB6\n4h8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f2a847d7e50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_performance(history)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Discussion**\n",
    "\n",
    "The standard AlexNet, trained from scratch obtains ~84% accuracy. The accuracy and loss plots shown above suggest overfitting. In the next section, we will try to overcome this problem through fine-tuning."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 2 : Fine-Tuning Alexnet\n",
    "\n",
    "We will follow the strategy as suggested in this paper - http://ieeexplore.ieee.org/abstract/document/7426826/?reload=true\n",
    "\n",
    "The basic idea is to perform the training layerwise. What this means is - consider a 5 layer CNN with layers {L1,L2,L3,L4,L5}. In the first round of training, we freeze layers L1-L4 and finetune layer L5 for some epochs. In the next round, we train layers L4,L5 for some epochs. In the next round, we train L3,L4,L5 for some epochs. Essentially, the training percolates to shallow layers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/rahulduggal2608/anaconda2/lib/python2.7/site-packages/keras/layers/core.py:577: UserWarning: `output_shape` argument not specified for layer mean_subtraction and cannot be automatically inferred with the Theano backend. Defaulting to output shape `(None, 3, 227, 227)` (same as input shape). If the expected output shape is different, specify it via the `output_shape` argument.\n",
      "  .format(self.name, input_shape))\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "____________________________________________________________________________________________________\n",
      "Layer (type)                     Output Shape          Param #     Connected to                     \n",
      "====================================================================================================\n",
      "input_1 (InputLayer)             (None, 3, 227, 227)   0                                            \n",
      "____________________________________________________________________________________________________\n",
      "mean_subtraction (Lambda)        (None, 3, 227, 227)   0           input_1[0][0]                    \n",
      "____________________________________________________________________________________________________\n",
      "conv_1 (Convolution2D)           (None, 96, 55, 55)    34944       mean_subtraction[0][0]           \n",
      "____________________________________________________________________________________________________\n",
      "maxpooling2d_1 (MaxPooling2D)    (None, 96, 27, 27)    0           conv_1[0][0]                     \n",
      "____________________________________________________________________________________________________\n",
      "convpool_1 (Lambda)              (None, 96, 27, 27)    0           maxpooling2d_1[0][0]             \n",
      "____________________________________________________________________________________________________\n",
      "zeropadding2d_1 (ZeroPadding2D)  (None, 96, 31, 31)    0           convpool_1[0][0]                 \n",
      "____________________________________________________________________________________________________\n",
      "lambda_1 (Lambda)                (None, 48, 31, 31)    0           zeropadding2d_1[0][0]            \n",
      "____________________________________________________________________________________________________\n",
      "lambda_2 (Lambda)                (None, 48, 31, 31)    0           zeropadding2d_1[0][0]            \n",
      "____________________________________________________________________________________________________\n",
      "conv_2_1 (Convolution2D)         (None, 128, 27, 27)   153728      lambda_1[0][0]                   \n",
      "____________________________________________________________________________________________________\n",
      "conv_2_2 (Convolution2D)         (None, 128, 27, 27)   153728      lambda_2[0][0]                   \n",
      "____________________________________________________________________________________________________\n",
      "conv_2 (Merge)                   (None, 256, 27, 27)   0           conv_2_1[0][0]                   \n",
      "                                                                   conv_2_2[0][0]                   \n",
      "____________________________________________________________________________________________________\n",
      "maxpooling2d_2 (MaxPooling2D)    (None, 256, 13, 13)   0           conv_2[0][0]                     \n",
      "____________________________________________________________________________________________________\n",
      "lambda_3 (Lambda)                (None, 256, 13, 13)   0           maxpooling2d_2[0][0]             \n",
      "____________________________________________________________________________________________________\n",
      "zeropadding2d_2 (ZeroPadding2D)  (None, 256, 15, 15)   0           lambda_3[0][0]                   \n",
      "____________________________________________________________________________________________________\n",
      "conv_3 (Convolution2D)           (None, 384, 13, 13)   885120      zeropadding2d_2[0][0]            \n",
      "____________________________________________________________________________________________________\n",
      "zeropadding2d_3 (ZeroPadding2D)  (None, 384, 15, 15)   0           conv_3[0][0]                     \n",
      "____________________________________________________________________________________________________\n",
      "lambda_4 (Lambda)                (None, 192, 15, 15)   0           zeropadding2d_3[0][0]            \n",
      "____________________________________________________________________________________________________\n",
      "lambda_5 (Lambda)                (None, 192, 15, 15)   0           zeropadding2d_3[0][0]            \n",
      "____________________________________________________________________________________________________\n",
      "conv_4_1 (Convolution2D)         (None, 192, 13, 13)   331968      lambda_4[0][0]                   \n",
      "____________________________________________________________________________________________________\n",
      "conv_4_2 (Convolution2D)         (None, 192, 13, 13)   331968      lambda_5[0][0]                   \n",
      "____________________________________________________________________________________________________\n",
      "conv_4 (Merge)                   (None, 384, 13, 13)   0           conv_4_1[0][0]                   \n",
      "                                                                   conv_4_2[0][0]                   \n",
      "____________________________________________________________________________________________________\n",
      "zeropadding2d_4 (ZeroPadding2D)  (None, 384, 15, 15)   0           conv_4[0][0]                     \n",
      "____________________________________________________________________________________________________\n",
      "lambda_6 (Lambda)                (None, 192, 15, 15)   0           zeropadding2d_4[0][0]            \n",
      "____________________________________________________________________________________________________\n",
      "lambda_7 (Lambda)                (None, 192, 15, 15)   0           zeropadding2d_4[0][0]            \n",
      "____________________________________________________________________________________________________\n",
      "conv_5_1 (Convolution2D)         (None, 128, 13, 13)   221312      lambda_6[0][0]                   \n",
      "____________________________________________________________________________________________________\n",
      "conv_5_2 (Convolution2D)         (None, 128, 13, 13)   221312      lambda_7[0][0]                   \n",
      "____________________________________________________________________________________________________\n",
      "conv_5 (Merge)                   (None, 256, 13, 13)   0           conv_5_1[0][0]                   \n",
      "                                                                   conv_5_2[0][0]                   \n",
      "____________________________________________________________________________________________________\n",
      "convpool_5 (MaxPooling2D)        (None, 256, 6, 6)     0           conv_5[0][0]                     \n",
      "____________________________________________________________________________________________________\n",
      "flatten (Flatten)                (None, 9216)          0           convpool_5[0][0]                 \n",
      "____________________________________________________________________________________________________\n",
      "dense_1 (Dense)                  (None, 4096)          37752832    flatten[0][0]                    \n",
      "____________________________________________________________________________________________________\n",
      "dropout_1 (Dropout)              (None, 4096)          0           dense_1[0][0]                    \n",
      "____________________________________________________________________________________________________\n",
      "dense_2 (Dense)                  (None, 4096)          16781312    dropout_1[0][0]                  \n",
      "____________________________________________________________________________________________________\n",
      "dropout_2 (Dropout)              (None, 4096)          0           dense_2[0][0]                    \n",
      "____________________________________________________________________________________________________\n",
      "dense_3_new (Dense)              (None, 2)             8194        dropout_2[0][0]                  \n",
      "____________________________________________________________________________________________________\n",
      "softmax (Activation)             (None, 2)             0           dense_3_new[0][0]                \n",
      "====================================================================================================\n",
      "Total params: 56,876,418\n",
      "Trainable params: 56,876,418\n",
      "Non-trainable params: 0\n",
      "____________________________________________________________________________________________________\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "alexnet = get_alexnet(input_size,nb_classes,mean_flag)\n",
    "alexnet.load_weights('../convnets-keras/weights/alexnet_weights.h5', by_name=True)\n",
    "\n",
    "print alexnet.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 45s - loss: 0.2274 - acc: 0.6335 - val_loss: 0.2005 - val_acc: 0.6913\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 45s - loss: 0.2000 - acc: 0.6945 - val_loss: 0.1931 - val_acc: 0.7100\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 45s - loss: 0.1968 - acc: 0.6915 - val_loss: 0.1815 - val_acc: 0.7362\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 46s - loss: 0.1969 - acc: 0.7020 - val_loss: 0.1773 - val_acc: 0.7475\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 45s - loss: 0.1918 - acc: 0.7130 - val_loss: 0.1755 - val_acc: 0.7500\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 45s - loss: 0.1877 - acc: 0.7205 - val_loss: 0.1697 - val_acc: 0.7588\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 46s - loss: 0.1853 - acc: 0.7135 - val_loss: 0.1651 - val_acc: 0.7750\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 47s - loss: 0.1881 - acc: 0.7205 - val_loss: 0.1602 - val_acc: 0.7762\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 47s - loss: 0.1826 - acc: 0.7260 - val_loss: 0.1587 - val_acc: 0.7863\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 46s - loss: 0.1806 - acc: 0.7350 - val_loss: 0.1572 - val_acc: 0.7863\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 45s - loss: 0.1813 - acc: 0.7355 - val_loss: 0.1505 - val_acc: 0.7913\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 44s - loss: 0.1818 - acc: 0.7315 - val_loss: 0.1518 - val_acc: 0.7850\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 45s - loss: 0.1763 - acc: 0.7330 - val_loss: 0.1427 - val_acc: 0.8025\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 45s - loss: 0.1731 - acc: 0.7470 - val_loss: 0.1743 - val_acc: 0.7412\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 44s - loss: 0.1782 - acc: 0.7340 - val_loss: 0.1438 - val_acc: 0.8050\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 44s - loss: 0.1667 - acc: 0.7440 - val_loss: 0.1393 - val_acc: 0.8125\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 45s - loss: 0.1624 - acc: 0.7630 - val_loss: 0.1571 - val_acc: 0.7675\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 44s - loss: 0.1651 - acc: 0.7550 - val_loss: 0.1352 - val_acc: 0.8150\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 44s - loss: 0.1615 - acc: 0.7640 - val_loss: 0.1568 - val_acc: 0.7725\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 44s - loss: 0.1636 - acc: 0.7625 - val_loss: 0.1324 - val_acc: 0.8250\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 44s - loss: 0.1828 - acc: 0.7305 - val_loss: 0.1404 - val_acc: 0.8037\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 46s - loss: 0.1660 - acc: 0.7600 - val_loss: 0.1321 - val_acc: 0.8187\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 46s - loss: 0.1546 - acc: 0.7695 - val_loss: 0.1244 - val_acc: 0.8337\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 46s - loss: 0.1428 - acc: 0.7915 - val_loss: 0.1331 - val_acc: 0.8137\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 45s - loss: 0.1422 - acc: 0.7930 - val_loss: 0.1202 - val_acc: 0.8425\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 45s - loss: 0.1363 - acc: 0.8025 - val_loss: 0.1694 - val_acc: 0.7525\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 46s - loss: 0.1372 - acc: 0.8015 - val_loss: 0.1392 - val_acc: 0.8050\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 45s - loss: 0.1293 - acc: 0.8110 - val_loss: 0.1203 - val_acc: 0.8300\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 45s - loss: 0.1284 - acc: 0.8170 - val_loss: 0.1315 - val_acc: 0.8125\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 46s - loss: 0.1233 - acc: 0.8305 - val_loss: 0.1177 - val_acc: 0.8488\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 44s - loss: 0.1215 - acc: 0.8275 - val_loss: 0.1586 - val_acc: 0.7750\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 46s - loss: 0.1106 - acc: 0.8430 - val_loss: 0.1182 - val_acc: 0.8287\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 46s - loss: 0.1080 - acc: 0.8445 - val_loss: 0.1109 - val_acc: 0.8475\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 46s - loss: 0.0962 - acc: 0.8680 - val_loss: 0.1095 - val_acc: 0.8500\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 46s - loss: 0.0903 - acc: 0.8810 - val_loss: 0.1077 - val_acc: 0.8525\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 45s - loss: 0.0848 - acc: 0.8855 - val_loss: 0.1039 - val_acc: 0.8488\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 45s - loss: 0.0792 - acc: 0.8915 - val_loss: 0.1032 - val_acc: 0.8625\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 45s - loss: 0.0717 - acc: 0.9070 - val_loss: 0.1067 - val_acc: 0.8575\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 46s - loss: 0.0734 - acc: 0.8985 - val_loss: 0.1183 - val_acc: 0.8375\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 45s - loss: 0.0719 - acc: 0.9045 - val_loss: 0.1061 - val_acc: 0.8538\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 47s - loss: 0.0675 - acc: 0.9130 - val_loss: 0.1209 - val_acc: 0.8400\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 45s - loss: 0.0660 - acc: 0.9110 - val_loss: 0.1034 - val_acc: 0.8500\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 45s - loss: 0.0592 - acc: 0.9245 - val_loss: 0.1042 - val_acc: 0.8600\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 45s - loss: 0.0566 - acc: 0.9280 - val_loss: 0.1064 - val_acc: 0.8512\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 45s - loss: 0.0552 - acc: 0.9300 - val_loss: 0.1062 - val_acc: 0.8500\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 44s - loss: 0.0507 - acc: 0.9370 - val_loss: 0.0976 - val_acc: 0.8675\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 45s - loss: 0.0497 - acc: 0.9330 - val_loss: 0.0977 - val_acc: 0.8712\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 44s - loss: 0.0450 - acc: 0.9475 - val_loss: 0.0982 - val_acc: 0.8662\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 44s - loss: 0.0396 - acc: 0.9540 - val_loss: 0.1015 - val_acc: 0.8600\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 44s - loss: 0.0405 - acc: 0.9520 - val_loss: 0.1021 - val_acc: 0.8612\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 45s - loss: 0.0389 - acc: 0.9480 - val_loss: 0.1022 - val_acc: 0.8538\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 46s - loss: 0.0400 - acc: 0.9540 - val_loss: 0.0985 - val_acc: 0.8675\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 44s - loss: 0.0365 - acc: 0.9570 - val_loss: 0.0976 - val_acc: 0.8638\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 46s - loss: 0.0325 - acc: 0.9615 - val_loss: 0.1124 - val_acc: 0.8450\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 45s - loss: 0.0332 - acc: 0.9575 - val_loss: 0.0922 - val_acc: 0.8725\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 47s - loss: 0.0256 - acc: 0.9720 - val_loss: 0.1066 - val_acc: 0.8562\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 45s - loss: 0.0298 - acc: 0.9675 - val_loss: 0.1017 - val_acc: 0.8600\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 46s - loss: 0.0281 - acc: 0.9660 - val_loss: 0.1047 - val_acc: 0.8550\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 45s - loss: 0.0246 - acc: 0.9720 - val_loss: 0.1093 - val_acc: 0.8512\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 48s - loss: 0.0218 - acc: 0.9770 - val_loss: 0.0990 - val_acc: 0.8688\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 45s - loss: 0.0251 - acc: 0.9690 - val_loss: 0.0947 - val_acc: 0.8662\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 46s - loss: 0.0229 - acc: 0.9725 - val_loss: 0.0970 - val_acc: 0.8725\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 45s - loss: 0.0212 - acc: 0.9745 - val_loss: 0.1003 - val_acc: 0.8688\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 46s - loss: 0.0196 - acc: 0.9765 - val_loss: 0.0998 - val_acc: 0.8625\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 45s - loss: 0.0190 - acc: 0.9795 - val_loss: 0.0883 - val_acc: 0.8888\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 47s - loss: 0.0141 - acc: 0.9855 - val_loss: 0.0890 - val_acc: 0.8825\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 44s - loss: 0.0128 - acc: 0.9880 - val_loss: 0.1007 - val_acc: 0.8700\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 47s - loss: 0.0140 - acc: 0.9875 - val_loss: 0.1030 - val_acc: 0.8550\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 45s - loss: 0.0107 - acc: 0.9885 - val_loss: 0.0931 - val_acc: 0.8850\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 46s - loss: 0.0124 - acc: 0.9885 - val_loss: 0.0989 - val_acc: 0.8775\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 47s - loss: 0.0151 - acc: 0.9850 - val_loss: 0.0977 - val_acc: 0.8750\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 44s - loss: 0.0124 - acc: 0.9885 - val_loss: 0.1240 - val_acc: 0.8438\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 47s - loss: 0.0134 - acc: 0.9860 - val_loss: 0.0907 - val_acc: 0.8812\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 46s - loss: 0.0125 - acc: 0.9870 - val_loss: 0.0911 - val_acc: 0.8875\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 46s - loss: 0.0101 - acc: 0.9905 - val_loss: 0.0997 - val_acc: 0.8700\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 44s - loss: 0.0092 - acc: 0.9910 - val_loss: 0.1077 - val_acc: 0.8600\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 44s - loss: 0.0105 - acc: 0.9890 - val_loss: 0.1052 - val_acc: 0.8638\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 44s - loss: 0.0106 - acc: 0.9875 - val_loss: 0.0903 - val_acc: 0.8775\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 46s - loss: 0.0085 - acc: 0.9910 - val_loss: 0.0949 - val_acc: 0.8812\n",
      "Epoch 1/1\n",
      "2000/2000 [==============================] - 45s - loss: 0.0083 - acc: 0.9925 - val_loss: 0.0871 - val_acc: 0.8962\n"
     ]
    }
   ],
   "source": [
    "layers = ['dense_3_new','dense_2','dense_1','conv_5_1','conv_4_1','conv_3','conv_2_1','conv_1']\n",
    "epochs = [10,10,10,10,10,10,10,10]\n",
    "lr = [1e-2,1e-3,1e-3,1e-3,1e-3,1e-3,1e-3,1e-3]\n",
    "\n",
    "history_finetune = []\n",
    "\n",
    "for i,layer in enumerate(layers):\n",
    "        \n",
    "    alexnet = unfreeze_layer_onwards(alexnet,layer)    \n",
    "   \n",
    "    sgd = SGD(lr=lr[i], decay=1e-6, momentum=0.9, nesterov=True)\n",
    "    alexnet.compile(loss='mse',\n",
    "              optimizer=sgd,\n",
    "              metrics=['accuracy'])\n",
    "    \n",
    "    for epoch in range(epochs[i]):    \n",
    "        h = alexnet.fit_generator(train_generator,\n",
    "                                        samples_per_epoch=2000,\n",
    "                                        validation_data=validation_generator,\n",
    "                                        nb_val_samples=800,\n",
    "                                        nb_epoch=1,\n",
    "                                        verbose=1)\n",
    "        \n",
    "        history_finetune = append_history(history_finetune,h)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAakAAAEbCAYAAABgLnslAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnXl8FEX2wL8vIQkJSSAJIUBCEgTkDocKyKEIive5Hisq\nKyre53quq6v+dHdFxWtddz3Xaz1Ql3VVRBEERQ5BkBsMR06SkJD7Dkn9/qieyUySSSaByWSS+n4+\n80l3VVd1daerX79Xr16JUgqDwWAwGDoift5ugMFgMBgMrjBCymAwGAwdFiOkDAaDwdBhMULKYDAY\nDB0WI6QMBoPB0GExQspgMBgMHRYjpAw+h4h8JyLXeLsdBkNHQEQSRKRORDrl+7xTXlRbEJEVIpIv\nIgHebou3EJF+IpLeyjL/EpEqESm2fiUisslTbTR0PkRkv4jM8HY7mkJEHhCRJ1pZps7qB8UOf+/x\nVBstOu2EVyOk0F8iwFSgDjivnc/t357na4GzgK/aUG6+Uirc+oUppcYd7YYZDF7ibGBxK8soIMmh\nP4QrpZ7xQNu6BEZIaeYAa4C3gKsdM0Sku4gsEJEUESkQke9FJMjKmyoiP1rpqSIyx0p3MkeJyO9E\n5AeH/ToRuVlEfgV+tdKeF5E0ESkSkfUiMtXheD8ReVBE9lhfZetFJFZEXhIRp4dfRD4TkTsaXqCI\nvCwiTzdI+6+I3OmQdBZWhxSR+0UkwzrfThE5pTU31KrDZoaYJyKZ1u9uh/xA67ozrXM956jJisj5\nIrLJuifJIjLLofpEEVlltW+JiERaZYJE5F0RybP+L+tEJLq1bTd0DKxnJ9n6f/5XRPo55D0nIjnW\n87FZREZY6WeJyHbr2UgXkd83UW+g9XyMcEjrLSLlItLb2u8FDAHWiEiUiHxulTkkIiuba7b1a+p6\nHhGRj0XkQ6t9G0QkySF/mPX+KBCRrSJyrkOey3eRdb4rrffQQRF50KHcCdY7o0hEshq+Mzo8Sqku\n/wOSgRuA8UA1EO2Q93dgOdAX/SBMAgKAeKAYuBTwByLQX08A3wHXONTxO+B7h/064GugJxBkpc0G\neqE/HO4CsoBAK+9eYDMw2NofbZ3vBCDDod4ooBTo3cQ1TgNSHfZ7AWVAjLXfDcgFegDHAmkOefHA\nQBf37l/A/7nIS7Cu9d9Ad2AUcBCYYeX/H7DaancU8CPwmJU3ASh0OLYfcKzD/U0GBgFB1v5frLzr\ngc+sdAHGAaHefsbMr9n+t9/2f26QPsN6JsdYfe5FYKWVNwtYD4RZ+0MdntcDwGRruycw1sV5Xwce\nd9i/GVjssH8Z8G9r+y/Ay1b/9AemNHM9dcAxLvIeAaqAC6167gb2WdvdrOf6fmv7FPQ7ZohV1tW7\nyNbPXgECgSSgEhhqlVsNXGFthwATvP0/b9Xz4e0GePuHNvNVARHW/g7gDmtbgHJgVBPlHgA+dVGn\nO0Lq5BbalQ+MtrZ3Aee4OG47MNPavgX4opk6U4Cp1vZ1wLcOeTOApdb2ICAbmAl0a6Gd/wIqrPYW\nWH//ZeXZOs8Qh+PnA69Z23uA0x3yZgH7rO1/Aguaub8POuzfZHu5AHOBVbZ7Z34d/4drIfU68KTD\nfg+rr8ZbL/BdwERAGpRLAeZhCbBmzjsT2OOwvwq40mH/HYeX+2PAImCQG9dTh/7AcuwTp1l5jwCr\nHY4VIBOYYr2LDjSo633gTy28ixKAWqCfQ9o64FJre4V13ihv/6/b8jPmPm3q+0YpVWDtf4AWKgC9\n0V/k+5ooNwDYewTnzXDcEZF7RGSHpcYXAOHW+W3naqoNoDvSldb2lcC7zZzzI+Bya3s2WsOxYTf1\nKaX2AncCjwI5IvK+o5mlCZ5WSkUqpSKsv3Md8hTO15oK9Le2+6M1tqbyWrq/2Q7b5UCotf0uWkv9\n0DIhPikda9zP4D790c8EAEqpMvQLP1Yp9R3wElq7yBGRf4qI7Rn4DXosKdUynU1yUf93QLBlDktA\na2yLAEREgNOAJdaxT6Gfx28ss/v9LbR9XIM+sdQhz+6cpLQUybSutb9jnkUqEIt+F3TH9XsAIMdh\n27FPXIvWNHdZ5u+zW2h7h6JLCykR6Y42151s2Wqz0C/nMSIyGshDq82DmiieDgx2UXUZWq220beJ\nY+zeONb4073AxdZDHYFW82127XQXbQB4DzjfsmsPA/7r4jjQAvhiEYlHf4F+6pBnF1IASqkPlVLT\n0F9pAE82U29zCFrg2IhHm2Ow/iY45CU45DV3zS5RSh1WSj2ulBoJTAbORX+IGHwPp+dDRHqgzcKZ\nAEqpl5RSxwMj0C/he630n5VSFwDRaNPvwqYqV0rVWXmz0R9vX1iCELQpPUUpdcg6tkwpdY9SahDa\nuer3LYzTNjkmZWHvD5YwjLOu9QC6fzgSb11vc++iZlFK7VVKzVZKRaOF7SciEtzaerxFlxZSaLvw\nYWA4+itqjLW9CphjfeX8C3hWtHu2n4hMsgb3/w3MFJGLRcRfRCJFZIxV7y/ARSISLCKD0V8yzREG\n1ACHrAHdP1lpNl4HHrfqQkRGi0gEgFIqE9iA1iA+VUpVuTqJUuoX4JBV3xKlVLFVXyJ6/Gu3tX+s\niJwiIoHoMboKtAmjrTxs3YuRaHPch1b6B8BD1oB1b+Bh6jXBN4C5VjtERPqLyLEtnUhEpovIKNFz\nRkrR9/VI2m5oHwJFO73Yfv7o52OuiCRZDgJ/AdYopdJE5HgRmSAi3dDPZyVQJyIBIjJbRMKVUrVA\nCdoU5ooP0GNPs9GmNRtnAV/adkTkbBGxCYgS9Hujrc/VcSJygXWNd1ltX4s20ZWJyH0i0k1EpgPn\nAB9Y76I3afpdBM0IRRG5wuYMAhShP5B9p094297ozR/a3fqpJtIvQX/V+KFV7GfRJqsCtH3X5uww\nBf1wFaHV8qus9Ci0yakI+AFtU3Yck6rFYWDVOs8b1vGZwD1otX6GQ/6DVloR+mHu71D+CqvOk9y4\n5oesYy9ySLsFeNFhf7R1jiL0F9z/gL4u6vsXupMVW78S4KCVZxuTus66rgPA3Q5lg4DnrfRM4Dks\nZxEr/3y0w0gx2gvSZtdfjosxP+C36LGKErTzyXOAn7efNfNr9pncbz2TtdbzUovljIN2hNnj8Bz2\nt9JnODwbB9EfNyFoR4Kv0B9jhdZzfGIL509GO2h0c0hbD4x32L/TamcJ2kT9YDP12YRjscPfZ628\nR9Da2wdW+s/AGIeyw9HvmEJgG3CeQ14QTbyLqB+T8nM41t5HrHuTY51vK3Cut//nrfmJdREeQUTe\nQH8J5Cilklwc8yJwJtpEdrXSX/uIyBnoF5gf8IZSar7HGurjiMg04F2lVGIby38J/E0ptaTFg1tX\nbwJasAYobVoxtJGW+oOIzEZ7hYF+Md6slNpi5aWgPzjqgBql1IT2arcvIiJ9gI1KqTgP1P0I2vnC\nmKDdxNPmvn8Bp7vKFJEz0f+wIWgX8H9a6X7oQdHTgZHA5SIyzMNt9Uksdf8O4LUjqOY76+cJmrPN\nG9zAzf6wD61JjwGeAF51yKsDpiulxhkB5RY90a7hhg6AR4WUUmoVWi11xflo7zSUUuuAniISg54j\nk6yUSlVK1aDHMM73ZFt9EetFVQDEAC+0tR6l1DOqmbGsI8RzqnrXocX+oJRaq5QqsnbXoj3CbAhm\n/NltlFLJSqmPvN0Og6abl88fi7PLZYaV1lS6+QJsgFJqF/Vuph0OpVQqepKi4chobX+4DufwVgpY\nKiK1wKtKqSPRug1HgFLqMW+3wdfwtpBqSJtMQyJivtYNHkUp5RNmS8stei56YqiNKUqpLNHhoZaK\nyE7LytGwrOlHBo/T2r7kbRNAJs5zaOKstEyc5wvY0l1ytDxJHnnkEVOXqcvp1wFwqz9Yc+VeRXuE\n2c3sSqks628uerKqSy2sI97/o1lXR25bV6irLbSHkHIZbBHtUmoLyjoJKFRK5aDdPweLDlAaiHYr\n/l87tNVg6Ii02B+sCdqfoqdB7HVID7FFYrAmw85CuzYbDD6BR819IvI+MB2IEpE09ByBQHQ0kFeV\nUotFRyzeg3ZBn4vOrBWRW4FvqHe53enJthoMDVEKpAMY+Vz1BxG5AasvoSdCRwIvW1EMbK7mMcAi\ny5TXDR0w9RvvXInB0Ho8KqSUUrPdOOZWF+lL0KFO2pXp06eburpIXa6EUGEhvPYavP46rFt31Jp0\nRDTVH5RSrzhsz0MHVW1Ybj8w1uMNbEBHfS6Odn2mLs/j0cm87YWIqM5wHYb24aWX4J//hD17YMIE\n+NvfYIwV0GrbNjjtNDj1VLjzTjjuOBARlI84ThwJph8ZPE1b+lJH8+47qiQmJpKamtrygV2QhIQE\nUlJSvN2MNrN2LdTVQWys3l6xAiorISoKHn8cgoNh1y749FO45x4IspaGe+EFePFF+OADGD4c3n8f\nZs2CGTPgkkvgtttgwQKY3aINoOtg+lHz+Hpf6uh0ak3KktpeaFHHx1fvTWUl3Hsv/Pe/0L8/pKVp\nbee00yA8HL78EsrK4Jln4PTTITERysvhjjtg6VL44Qf4/ntIcIi9XlQEb7wBb70Fd90Fc+c6n7Or\na1K++qy0F+b+uE9b+pIRUl0UX7w3SsGZZ0KPHnq8KCKi8TGHD8NvfwuffQYvvwzXXadNe0uWaEF2\nySUQE9O68xoh5XvPSnti7o/7GCHVON08PC7wxXvz2Wfw4IOweTN0a8ZQXV0NGzfCJFdL3bUSI6R8\n71lpT8z9cR8jpBqnm4fHBb52b6qrYdQoPZ50xhnte24jpHzrWWlvzP1xn7b0JW9HnDAY3GLBAhg0\nqP0FlMFg8C5GSPkoN910E3/+85+93YyjQkWF6zyl4LHH9Lylv/+9/dpk6Dp0pr7UGTHmPi8xcOBA\n3njjDWbMmOGV83eUe/P223qcad++ejdxG2lp2tsuNRW++AL69vVOG425r2M8K64wfcl3MOa+TkJt\nba23m9AupKfrOUwREfDJJ/XplZXw5z/D+PEwerR2GfeWgDL4Nl2lL3VmjJDyAnPmzCEtLY1zzjmH\n8PBwnn76afz8/HjzzTdJSEhg5syZAFx66aX069ePiIgIpk+fzo4dO+x1zJ07lz/96U8ArFy5kgED\nBvDss88SExNDbGwsb731ljcurUl+/RVqapzTlIJrr9VRHZ54QkeBAPjpJxg5EjZsgPXr4dFHISSk\n3Zts8BG6Wl/qihgh5QXeeecd4uPj+fLLLykuLubSSy8F4Pvvv2fXrl18/fXXAJx11lns3buXgwcP\nMn78eK644gqXdWZnZ1NSUsKBAwd4/fXXueWWWygqKnJ5fHuhFJx8Mtx/v3P6woWQm6vTzz0XsrLg\n1VfhnHPg6adh0SIYONA7bTb4Dl2pL3VZjtY6Id786ctojKv0+vyj82sLiYmJatmyZUoppVJSUpSf\nn59KSUlxeXxBQYESEVVcXKyUUurqq69WDz/8sFJKqRUrVqiQkBBVW1trP75Pnz5q3bp1zVx7Gxve\nSpKTlYqJUSohQalFi3RaRYXeX7Gi/rj585Xy968/piNh3SuvP+ee/rW1H+ljvNOPlOo6fakz0Ja+\n1Klj97WE6mBjnXFxcfbturo6HnzwQT755BPy8vIQEUSEvLw8wsLCGpWNiorCz69eMQ4JCaG0tLRd\n2t0cq1bB9OnarHfuuZCSAocO6fGmk0+uP+6222DmTB3iyOB7mL5k8BRdWkh5E2lijQjHtPfff5/P\nP/+c5cuXEx8fT1FREREREbYvXp9h1SqYOlVHf1i8WMfUW7wYNm1yPi442AgoQ9voKn2pq2LGpLxE\n37592bdvH1BvcnWkpKSEoKAgIiIiKCsr4w9/+EOTnbGjYxNSACecAB99pNdrGjzYu+0ydB66Sl/q\nqhgh5SUeeOABHn/8cSIjI/n0008bdZo5c+YQHx9PbGwso0aNYvLkya2q35udcNMmyM7WjhFZWdqN\n3BF/f++0y9A56cx9yWAm83ZZPHVv9u7VGtOAAXpJjffe0xHIfRkzmdf0o+Yw98d9zGReg1eprIRL\nL9VzmyZOhOuvrzf1GQwGQ1vwuOOEiJwBPI8WiG8opeY3yO8FvAkMAiqAa5RSO6y8FKAIqANqlFIT\nPN1eQ9uortaLBSYmak+9w4e10DrvPG+3zNAalAJj3TJ0JDxq7hMRP+BXYCZwAFgP/FYptcvhmKeA\nEqXU4yIyFPi7UupUK28fcJxSqqCF8xgzRStpy71RCvLyIDraOb24GC66CMLC4N//7nwRIrqSua+0\nVNGjR6N004+awdwf9+mI5r4JQLJSKlUpVQN8CJzf4JgRwHIApdRuIFFEbK9BaYc2Gtzkgw+gXz8d\n9LWwsD79llv0GNQnn3Q+AdXVyMvzdgsMBmc8LQBigXSH/QwrzZHNwEUAIjIBiAdsM/EUsFRE1ovI\nPA+31dAMNTXwpz/B++9DSYkec8rP17H2li+Hv/3NeO11Bg4d8nYLDAZnOsJk3ieBF0RkI7AV2ATY\nQhdPUUplWZrVUhHZqZRa5a2GdmXeekuPN116qf7dcw/85jd6LOrxxyE01NstNBwNjCZl6Gh4Wkhl\nojUjG3FWmh2lVAlwjW1fRPYD+6y8LOtvrogsQpsPmxRSjz76qH17+vTpTJ8+/Wi03wBUVcH//Z/z\nchrz5+txqNRU+N3vvNc2T7BixQpWrFjh7WZ4BSOkDB0NTztO+AO70Y4TWcBPwOVKqZ0Ox/QEypVS\nNZZJb4pS6moRCQH8lFKlItID+AZ4TCn1TRPnMY4TraQ19+aHH+D3v9dLZzhSUwNlZdCrlwca2IHo\nSo4TL7yguP32RummHzWDuT/u05a+5FFNSilVKyK3ogWMzQV9p4jcoLPVq8Bw4G0RqQO2A9daxWOA\nRSKirHb+uykBZfA8q1bBSSc1Tg8I6PwCqqthNClDR8PjnnNKqSVKqaFKqSFKqSettFcsAYVSaq2V\nP1wpdbFSqshK36+UGquUGqeUGm0r21kYOHAgy5cvP6I63n77baZNm3aUWuSaVatgyhSPn8bQAfBF\nxwlf6kuG1mPcu30YpZTH44rV1cHq1UZIdRW6qibVHn3J0DaMkPICtiWvzz33XMLDw3nmmWdYt24d\nU6ZMISIignHjxrFy5Ur78W+99RaDBg0iPDycQYMG8cEHH7Br1y5uuukm1qxZQ1hYGJGRkR5p6/bt\nevJuTIxHqjd0MHxNSPlSXzK0kdauktgRfxzBiqLeIjExUS1fvlwppVRmZqaKiopSS5YsUUop9e23\n36qoqCiVl5enysrKVHh4uEpOTlZKKZWdna127NihlFLqrbfeUtOmTWvT+Zu7N3V1Si1YoFRhoVIv\nv6zU3LltOkWngS60Mm9Sksvr77B05L5kcKYtfakjzJPyGvLY0VHv1SNt8+xRlkfQe++9x9lnn83p\np58OwMyZMzn++ONZvHgxv/nNb/D392fr1q3ExcURExNDjIfVmqVL4YEH4OuvtWOE1SxDF6CtY1JH\noy+1tR9Bx+1LhiOnSwupI+kUR5PU1FQWLlzI559/DugOd/jwYWbMmEFISAgfffQRTz/9NNdccw1T\np07lmWeeYejQoR5rzxNPwOuv6zBICxfqOVKGrkFeXtuCzJq+ZPAUZkzKSzgO0g4YMIA5c+aQn59P\nfn4+BQUFlJSUcN999wFw2mmn8c0335Cdnc3QoUO5/vrrG9VxtPj+e71Q4ezZehXdRx+FY4896qcx\ndFD8/fXcN1+io/Ylw9HBCCkv4bjk9ZVXXsnnn3/ON998Q11dHZWVlaxcuZIDBw5w8OBB/ve//1Fe\nXk5AQAChoaH4+el/W0xMDBkZGdTU1ByVNh04oBcq/MMfoFs3CA+HRx4xSzd0JaKifM95oiP2JcNR\npLWDWB3xhw86Tnz22WcqPj5eRUREqAULFqiffvpJnXzyySoyMlL16dNHnXPOOSo9PV1lZWWpk08+\nWfXq1UtFRESoU045Re3cuVMppVR1dbU655xzVGRkpIqOjm7V+Rvem5deUioqSqk//EGpmpqjdpmd\nArqQ48S4cUpt2NDk9XdYOlpfMrimLX3JLB/fRXG8N089BW++CV98AYMHe7lhHZCOEBbJjcVDZwP3\nW7slwM1KqS3ulHWoQ512muLuu52dZUw/ah5zf9ynw4VFMnR8Xn8dXn0VVq6E2IaLqBg6BNbioS/h\nsHioiHymHBYPRQdlPkkpVWQJpVeBSW6WtdO7t++Z+wydGzMm1cX529/g7beNgOrgtLh4qNLhxYqs\n3bXUr9vmzsKjdnxxTMrQuTFCqgtz6BCkpMCECd5uiaEF3Fk81JHrgK/aUrZ3b9+M32fovBhzXxdm\n5Uodky8gwNstMRwtROQUYC4wtS3lN258lG3btEfnKaeYddkMR8bRWJvNCKkuzIoVYN5BPkGLi4cC\niEgSeizqDKVUQWvK2njm9blcNCOBYcPMs2E4chouQPvYY4+1ug5j7uvCGCHlM6wHBotIgogEAr8F\n/ud4gIjEA58CVyml9ramrCPLUr/izTfhzjvh559hzZqjfi0GQ6vo1JpUQkKCmUnuggEDEkhNhfHj\nvd0SQ0so9xYPfRiIBF4W/dDXKKUmuCrr6lxL9izhxt/eyB13wGWXQW0tREaaftQcCQkJ3m5Cp6ZT\nz5MyNOb++2HRIjj7bPj1V/jyS2+3qOPTEeZJtQciosL/Gk7uvbkE+gcC2rHm+OMhLQ1CQrzbPoPv\n05a+ZMx9XYjKSj1p9557tHA64wxvt8jQ0RjWexir0lbZ9xMTYeJEHcfRYPAGRkh1If7zHxg3Dq6/\nXmtRt97q7RYZOhpnDj6Tr/d87ZR2003wj394qUGGLo8RUp2UnBx49lnntFdf1QLKhhlmMDRkYuxE\nNmRtcEo780w4eBDWr/dSowxdGo8LKRE5Q0R2icivInJ/E/m9ROQ/IrJZRNaKyAh3yxpcs3ixjmie\nbk3j3L0bdu2C813GGjAYYEzfMWzO3uwUi87fH37/e3j8cS82zNBl8aiQcogbdjowErhcRIY1OOxB\nYJNSagzwO+DFVpQ1uGDtWr2q7ptv6v0//xluuMFM3DU0T7/QfogIWaVZTunXXw+bNsG6dV5qmKHL\n4mlNyp24YSOA5QBKqd1AoohEu1nW4IK1a2H+fB1Adu1a+PZb7TBhMDSHiJAUk8Tm7M1O6d27w8MP\n658j1jJOBoPH8LSQcidu2GbgIgARmYCeHR/nZllDE5SUwJ49MGeODhx7wQV6CfiwMG+3zOALjIkZ\nw5acLY3S587VQmnhQr3/+ecwaBBs3NjODTR0KTrCZN4ngRdEZCOwFdgE1La2kkcffdS+3TAUR1dj\nwwYYOxYCA7UH3/PP6xeMwT2ORrwxX2ZMzBi+3vt1o/SAAPjkE5g1C8rL9Zy7Cy6AN94wk8INnsOj\nk3lFZBLwqFLqDGv/AfQM+SYXXbOO2Q+MBka5W9ZM5nXmr3/Vyy0sWKD3Dx/Wy8EbWseNX9zI06c9\nTXj38C4zmVcpxaasTVy56Eq237y9yeO++07PsXvqKbjwQj2tISMDgoPbucEGn6MjTuZ1J+ZYTxEJ\nsLbnASuVUqXulDU0zdq1MGlS/b4RUK0noziDT3d+SmhgqLeb0u6MiB7BvoJ9VB6ubDL/lFMgNRVu\nvx3i43VEikWL2rmRhi6DR4WUUqoWsMUN2w58aIs5JiK2GTvDgW0ishPtyXdHc2U92d7OgFKNhZSh\n9axJX8PkAZO7ZMy6oG5BDI4czI7cHS6P6du3fp7dtddqBx1PGDNqauCDD45+vQbfwcTu62Rs2AAX\nXaS/dLvg+/WocdeSu+gb2pf7p97fpWL32frRFf+5glMHnsrccS0PZlZVaW3qrLPgySd1UNq1a+H7\n76F/f7j66ra3aft2XXd5uXmeOwMd0dxnaEeU0hN4//jHztehd+ftJrcst93OtzpjNZMHTG6383U0\npidMZ+GOhW4dGxSkl31ZvhxmzNCC6fbb4cABuPtuqKhoezsyMnTMyZycttdh8G2MkOpEfPYZ5OZq\n80tn48HlD3LjlzceUR0rU1by6Y5PWzyuoqaCbQe3cXz/44/ofL7MnDFz2H5wO2vS3VtQKioKli2D\n2bO1FrVxI7z0EpxwwpGNV2Vk6L8pKW2vw+DbGCHVSaio0FrUs892TkeJ/QX7WZy82OVLs7nxE4Ca\n2hrmfT6PV35+pcVzbTiwgZHRIwkO6LruakHdgnjopId4ZMUjbpcJD4d58+CYY+rTrrmmPuqJjYwM\nKCx0r05bWC8jpLouRkh1Em6/XdvuZ83ydks8w/7C/Tx+yuPc9+19NBx/rKmtYfQ/RrMuw3XMntc2\nvkZYUBgbszY2Kt+Q1eld29Rn4+qxV5Ocn9woKnprOP98+OUXZyEzezY895x75TMydHgvI6S6LkZI\ndQLeew9WroRXWlYSfJLCykIO1x3mrkl3kV+Rz/L9y53yM0syqVN1LFizoMnyJVUlPP7947x27msE\n+AeQXpze5HE2uvp4lI1A/0BeP/d1rlp0FR9u+7BNdQQFaaH06qt6f+tW+OEH2Ly5+XI2MjJgyhTY\nv79Npzd0AoyQ8nGqq7UW9fHH2tzSGdlfsJ+BvQbi7+fP5aMu58tk5+WE04vSSYpJYvn+5ewvaPw2\nW5y8mHF9xzG+33jG9R3HpqxNLs/1fer3rE5fzUkJJx316/BFZh4zk2/nfMt9S+/jq+Sv2lTH3XfD\na6/pKPz/+IcWWq0RUlOnGk2qK2OElI+zbp0eAxgzxtst8Rz7C/czMGIgALMGzWLpvqVO+enF6Qzv\nPZzrxl/H82ufb1Q+qzSLwZGDARjfbzwbs5oONrc1ZyuXfHwJ71/0Pn1D+x7lq/BdkmKSeOTkR3ht\n42ttKp+QAI89Bmc/8AFvbf4XTz6pI6K4My6VkQHTphkh1ZUxQsrHWbYMZs70dis8i02TAjiu33Fk\nFmdyoOSAPT+9KJ0B4QO4feLtvL/tfV5e/7LTuFN2aTYxPWIAS0hlNy2krvnfNcw/dT6nDTrNg1fj\nm1w84mKW7V/GofJDbSp/441QHfUz/Y7bwIABMGoUbGkcw9aJkhI9mXfcOEhLg7q6Np3a4OMYIeXj\nLFum56Y7v70mAAAgAElEQVR0ZvYX7ueYCO0y5u/nr01Q+76156cXpzOg5wD6h/Xnx2t+5PWNr3PD\nFzfY83PKcogJ1ULKlbmvoKKAXXm7mD16toevxjfp2b0nZw4+k4Xb3Zs71RA/P5h+ZgFjT8wHtObf\nkskvIwPi4iAkRJuys7PbdGqDj2OElA9TWqoXops61dstaZqM4gwO1x1uVZmqw1VklTgvuLevYJ9d\nkwI47ZjTnEx+6cXpxIXHAXBs1LF8dcVX/Gfnf+z5OaU5dvNdYq9EymrKyCl1nh26Km0VE2MnEugf\n2Kr2diWuSrqKd7e82+byZbUFlNUWADpKv7tCCiAx0Zj8uipGSPkwP/wAxx0HPXp4pv7Kw5X8/ae/\nU1vX6pVTAB1aZ9HO1s3kfODbBxjzzzEkH0q2pzmOSYElpPYutZv0bOY+G3169KG0upSKGh3qIKcs\nx27uExGtTWU7a1MrU1dycsLJrbvALsasQbPYW7CXXw/92qbyBZUF5Fe0XpMCGDiweSG17eA29hWY\nFRg7I0ZI+TCeHI+qratl9qezue/b+3h2zbNtqmNfwT5+yvypxePKqssAPfb0zpZ3uGvSXZz+3ulk\nl2ajlCKlMIXEXon24wdGDCQ0MJRtB7cB9eY+GyJCbHgsmSWZgNakbOY+0ONaDdu1MnUl0xOnt+k6\nuwoB/gHMGz+P59a4OcmpAQUV9UJq9Ggdl+9wM4p2err7mtSrP7/K+1vfb1O7DB0bI6R8mO++89x4\n1B1L7qC4qphNN2ziqdVPsf1g02sLAaQWptoFho3q2moyizP56UDzQuqX7F+IfjqaV39+lYe+e4jb\nJ9zOH6b9gSuTruTa/11Ldmk2YYFhjZbMOCnhJH5M/5GKmgqKq4rp06OPU35ceBwZxRkopThYdtAp\n/+xjz2bRrnoNr7iqmF15u5gQO6HF+9LVuX3i7Xy0/SOyS1s/QOSoSYWG6lWjf21GKcvIgAHWt0di\nYvNzpcqqy+x1GzoXRkj5KHV1sGOHtu0fbcprynlj0xt8eumnHBt1LH+d+Vd+99/fUaeadq+a/+N8\npr45la05W+1pGcUZRARHsClrU7PmwnUZ65gaP5W/r/87y/Yt4+7JdwPwx2l/ZGfuTv71y7+cTH02\nJg+YzOr01WSWZBIbFoufOD/KsWGxZBRnUFBZQEhACN27dbfnTYufRk5pjt1stSptFcf3P56gbkHu\n36QuSp8efbh81OW8uO7FVpfNr8insLLQ/hydcIL+0HJFa8x9ZTVGSHVWjJDyUTIyICJCf5Eebbbk\nbGF47+H07N4TgGvHXUugf6BLc8rK1JVcf9z1nPnvM0krSgMgpTCFUX1G0Te0LzvzXC8DtjFrI+cN\nPY91161j3XXr7BpTULcg/jzjzzy64lEnpwkbNiGVXuRs6rNh06QamvpAewhePOJiPtr2kW5/ihmP\nag33TL6HV35+heKqYrfL1NTWUFFTQWhgKEWVRQDcdhs8/bSekK6UnuhbUFBfxlFIDR6sJwO7orS6\nlILKAtcHGHwWI6R8lORkGDKkcXpZdRnPrXmuxfh0zbEpaxPj+42374sI80+dz0PLH2q0WuvBsoNk\nFmfy15l/5eIRF/PKBh2bKbUwlcReiUyIncD6zPWuz5Wtz9W9W3cSeiU45V026jKSYpKaFFLDeg/j\nUMUhNhzY4OQ0YSMuPI7M4kynOVJOdY+8jIU7FrLhwAbe/OVNfjP8N83fFIOdgREDOfWYU3nrl7fc\nLlNYWUiv7r3oHdLbrvGceCIceyy88w787W9wxx3wpz/Vl3Eckxo4EIqKIN+FsmQ0qc6LEVI+iish\n9V3Kd/z+m9+7HR2gqLKokRlvY9ZGxvUd55Q2LWEaSTFJ3Lr4Vi786EJmf6rnE32f+j1T4qfg7+fP\nSQknseWgnqGZUphCQs8ETuh/AusPaCHVcByjpraG7bnbSYpJarJtfuLHossWceekO5vMmxQ3iYU7\nFroUUhklGeSU5TQZPeLEASdSWFnIGe+dwevnvs7omNGubpGhCW6bcBsv/fSSSxNwQwoqC4gIjiAy\nONJJmDzyCDz0EPz5z7B6NXz0kY7v98IL0Lu3XgIE9DyrpCTXHoFmTKrzYoSUj+JKSK1MWcnloy7n\nj8v/2GQcu4ac9+F5jbz3bNpNQ5467SmyS7M5Z8g5rExdyebszU6msqSYJLbkWEKqSHvknRB7Aj9l\n/sSXv35J3LNxfLC1fi3wnXk7ie8Z38gpwpEBPQc0MtfZmBw3WWtSTZj7bGNSOaU5TWpSfuLHg1Mf\n5LnTn+P8Yee7PL+haaYMmEJoYKjbEdILKgqI6B5BRHCEkzCZMgXOOw/efVdH8X/0UTj7bC2kvv3W\nefHO5tzWjSbVeemEKw91DZKTdQdvyMrUlTwz6xnG9R3HnV/fyWe//cxlHRnFGazNWEt2aTZ3n3g3\nIkJNbQ07cnc0qd0M6z2ML2Z/AUBueS4L1izgl+xfeO1crbUdE3GMfXA8tTCVxDGJjOs7jh25O7j6\ns6t54YwXuPPrO5meOJ1+Yf3YlLWpkcbWGmyRyl1qUsUZTtEmGnLTCTe1+dxdHRHhtgm38eJPL3Lm\nkDNbPD6/Ip+I4Ah6de/VaOzIFiEd4Prr9bN9110QH+9cx9ix8OOPTddv06SUUkhnW5a6i+NxTUpE\nzhCRXSLyq4jc30R+uIj8T0R+EZGtInK1Q16KiGwWkU0i0vKEmy5EcrIeTHakpKqEHbk7mBA7gd+N\n/R2r0lY1Ozb18faPuWL0FQjCj+m69+/I3UFir0R6BDY/Q/iG427gi1+/IKUwxa51+Ykfo/qMYkvO\nFru5LzggmAuHX8jbF7zNLRNu4frx1zPv83kopdiYtbFJjc1dJsROwE/8mtSk+ob25VD5ITKKM5rU\npAxHzuWjL2dX3i7uWnJXo7HKhhRUak0qsntksxpPt256ramGAgpa1qSqa6sprylvzSUYfACPCikR\n8QNeAk4HRgKXi8iwBofdAmxXSo0FTgEWiIhNw6sDpiulximlzCQWi9paPWekoZD6Mf1Hju9/PN27\ndadPjz74iZ99HGh95nonUxvAR9s/4rKRl3HtuGt5Y9MbAG4LjojgCOaMmcPkAZMJ8A+wpyf1SWJj\n1kaySrPswuPfF/2bs4acBcDDJz9MfkU+D3z7ABuzG499tYawoDCeP/15hvVu+EhpD76Y0Bg2ZW9y\nqUkZjozu3bqzYd4GMkoyOP7V4/kl+xeXx9rMfQ3HpFrDqFGwe7f2BmxIWXUZEd0jmq1bKcUD3z5w\nRE5FhvbH05rUBCBZKZWqlKoBPgQaDgAoIMzaDgMOKaVs89ClHdroc6Sl6UHlkBBYuH0hCc8nsDJl\nJStSVji5Uo+MHsn2XD0J98NtH/Ls2vqxp5TCFPYW7GXGwBnMGTOHRTsXkVOa43I8qin+MvMvvHm+\n89rgY/qO4as9XxEdEt1kHLxA/0A+v/xzPv/1c1anr2Zcv7YLKYDbJt7mNAfKkbjwOHbk7jCalAeJ\nColi4cULuX/K/cx6d5bL+VMFlQVEBkc2GpMC2Jy92UkTq6mtabKOkBC97EdDV/Q6VUfl4Upiw2Ob\nFVJVtVXM/3E+ZTVlbl6doSPgaQEQCzgug5phpTnyEjBCRA4Am4E7HPIUsFRE1ovIPI+21IdIToZj\nhlZww+c38OCyB7l38r1c8vElfLLjE05OrBdSo/qMskeKWH9gPRuzNtrntizcvpALh11IgH8AMaEx\n3DrhVo596Vje2fyO20IqJCCE/mH9ndKSYpL4bv93TmGMGhIVEsWSK5dwz4n3EBkc2cqrd5/YsFjq\nVJ1ZG8rDiAhXjbmKn+b9xEPLH6KwsvFCUQUV9d59Dcekrvv8OqfVlge9OMilsGnK5FdeU05wQDBR\nwVHNCinbs19SVeLupRk6AB3BceJ0YJNSaoaIDEILpSSlVCkwRSmVJSLRVvpOpdSqpip59NFH7dvT\np09n+vTp7dB0z/LO5nfwF3+uSLrCKX3Vrp1sO/FSYqtHsfGGjYQHhdOnRx/mfT6PSXGT7MeNjB7J\nxqyNHK47zMasjSTFJPFj2o+cOUQvuTD/1Pn2Y5+Y8QR3n3g3S/ctZcqAJjwy3GR0n9HU1NU0mvPU\nkPie8cw/bX6zxxwptsjorTX3rVixghUrVnigRZ2bxF6JnJx4MouTFzda8iS/Mp8R0SOaNPelFqba\nI9+X15STXpzOgZIDTX7AjB0Lv/wCV11Vn1ZWXUaPgB4tmhJtwqmkuoR+9GvrZRraGU8LqUzAcQg0\nzkpzZC7wVwCl1F4R2Q8MAzYopbKs9FwRWYQ2H7YopDoLa9LXEB4UzhU4C6lXcq9iWsh1vH/R7XZP\npktHXsp5Q89zMn2N7DOSd7e8y87cncSGx3L+0PNZmbqSIVFDSC9Od9K6QI8zXTry0iNqc8/uPUns\nlUhiz8QjqudoEBceR3hQuEtzoCsafuQ89thjR7llrUdEzgCeR1s/3lBKzW+QPxT4FzAeeFAp9axD\nXgpQhB7jrfHk+O4FQy/gv7v+20hIOWpSjoKkvKac3PJcskq1kLIJq4NlB5usf8wYWLDAOa2spowe\ngW4IqWotpFoTKcPgfTxt7lsPDBaRBBEJBH4L/K/BManAqQAiEgMcC+wTkRARCbXSewCzgG10IXLK\nciitLnVKKyiqIU92MHvYvEautg1fxrYxqXWZ6zih/wmcnHAyK1NXsnD7Qn4z/Dd08/PMN8rYvmPt\nixR6k7jwuE4xHuWmA9Ih4Dbg6SaqaDcHpPOGnsc3e79p5O1n9+5rIEjSi/RogE042YSVKyFlm9Dr\n6PvQak3KmPt8Co8KKaVULXAr8A2wHfhQKbVTRG4Qkeutw54AJovIFmApcJ9SKh+IAVaJyCZgLfC5\nUuobT7a3PdmY1fQS5o5kl2ZTWlMvpN58EwaesJuQmnjOmBnSYvmokChCAkJYtGsRE2InMCluEltz\ntvLO5ne4bORlR9T+5nj1nFcbmSi9wag+o5gYN9HbzWgSEYkQkaZDbTSmRQckpVSeUupnoKnFL9rN\nASm6RzRJMUks27eMH1J/sC8+WVBhOU408MBLLUoFILtMe6G2pEn176+DK+c4rFnpriZl06CMJuVb\neHxMSim1BBjaIO0Vh+0s9Bdiw3L7AQ/E+PY+B0oOcNK/TqL0wdJmj8suzXZyTFiwAG57Ygu7JYme\nPd0716g+o1iyZwkPTXuI4IBgxvcbz578PUyN99xyvtE9oj1Wd2sY1WcU717Y9pVkjzYisgI4D93v\nfgYOisiPSqnft1C0KQek1mhENgekWuBVpZR7MbPayAXDLuDmxTdTWl1KdEg0Fw2/yCksUkFFgX3S\nbVpRGkMih7itSYlobWrLFuhr+cM4alJ78/e6bJfN3Gf7a/ANjHu3F0gtTKW8przZ+RpKKa1JWea+\nwkJITYXKXptdxrpripHRI/ETP8b21fL+nGPP4eqxV+Pv539kF2FoCz2VUsXARcA7SqmJWKZuDzNF\nKTUeOAu4RUQ894UCXDH6Cq5Kuork25LJLs0mtyzXPk8qOCAYEaHisF41Oa0ojYlxE53GpOJ7xpNT\nmuOyfpuQsuGkSVW2bO4zmpRv0RG8+7ocaUVpKBRVtVUuB/VLqkuoOFxhF1Lr1+ul4rflbuGm490P\n5zMyeiSj+4wmOCAYgHsn33vkF2BoK91EpB9wKfDHVpRzxwHJJW11QGqrl2xMaAxPzHgC0IF8v0v5\njqraKnuMRptZLiQghNSiVCbGTuSTHZ+glCKrNIsxMWM4WN60JgVaSK1cWb/v9phUtRmTam+Ohqes\nEVJewGaHr6ipcCmkbJEibEJq7VqYNAney9nCmJgxbp/rwuEXMrR3vbXVxDXzKv8HfA2sUkqtF5Fj\ngGQ3ytkdkIAstAPS5c0cb/8ni0gI4KeUKnVwQHLprni0vWSnDpjKF79+Qa/uvezPnk2YxIXHkVaU\nxpykOQT6B1JYWciBkgNMipvE0n1LXdaZlKSX9rBRVuOekCquKsZf/I25rx05Gp6yLZr7ROQ2EYlo\ndc0GJ5bvX25foda2MGBzccZs6yDZZsevXQsjjs+jtLqU+J5NBDZzQe+Q3pyUcNIRtNxwtFBKfayU\nSlJK3Wzt71NKtbiQlTsOSCISIyLpwF3AH0UkzfKO9aoD0pT4KSxOXuw05ymiewQFFXpCb2phKvE9\n4+kb2pfs0ux6TcrFmBTAyJE6PFKNFZiirNpNF/SqEvqF9TPmPh/DnTGpGGC9iCy0gsWaT/FWUl1b\nzVn/Psse28wmpGx2+abILs1mcORgSqtLUUoLqeDELSTFJBltyEcRkaesgMoBIrJMRHJF5Ep3yiql\nliilhiqlhiilnrTSXlFKvWpt5yilBiileimlIpVS8UqpUqXUfqXUWMv9fLStbHsxIXYCRVVFRHSv\n/861CZPauloySzIZ0HMA/UL7kVWaRVZJFmP6Ni+kQkJ0ANrdu/W+u5pUSXUJsWGxRpPyMVoUUkqp\nh4AhwBvA1UCyiPzFig5hcINNWZuoqq2yx9FLK0rDT/waaVLlNeWkFKYAWkj1rB1MUXkpe/boZeIP\n1LbO1GfocMyyHCfOAVKAwUCnHiQMCQjhuH7HERHsLKQOVRwiuzSbiO4RdO/WnX5h/UgtTKW4qphj\nIo6hpramWUuDo/OETZPqEdCDmtoalxHZS6pL6B/W32hSPoZb3n1Ku6FlW7/DQATwiYg85cG2dRpW\np6+mm183exy9tKI0BvYa2KgTfrz9Y67937WAFlIpm46h4nA5t92uOPFE2JKzhdF9zAqyPoxtDPhs\n4GOlVJE3G9NeTI2f6qRJjYkZw9J9S0krSrObrvuF9uOX7F/s0fv79OjTrDaVlARffQUHD9ZrUiJi\nd3FviuKqYq1JGccJn8KdMak7RORn4CngR2C0Uuom4DigRXu6AVZnrOaCYRewLXcbJVUlVNVWERce\nR0WNs7lvb8FeNmZttLycsknbHkv3bt3Zn17BSSdBZklmizHxDB2aL0RkF7rvLLNiUja/EFMnYO7Y\nucwZM8e+f824a1i+fznfpXxnf577hvZlY/ZG+7zAloTU5ZdDQQEceyys+qnMvv5Zcya/kqoSYsON\nuc/XcEeTigQuUkqdbg381gAoperQZguDAz9l/mQfcwI932l1+mrmjZ/H9oPb7V+PIQEhjTSpvQV7\nKawsJKUwhT1Z2YTSl7DuPVixupQbb4TCykJ6de/V3pdkOEoopR4AJgPHW/2ojMZL13Q6RvYZyRmD\nz7DvhwWFccNxN/DXVX8lPtxZk+oXpgO/xoTGcLDsIEoptuRsaVTn4MHwxRewZg3sSC4j2N8NIWWN\nSRlzn2/hjpD6CrD/162B34kASqmdnmqYr/L82ud5f+v79v20ojRq62qZOXAmueW5bM/d7lpI5e8l\nOiSajVkbScnLZuKIvoQGhlJRW4qfnxFSvo6IBABXAh+JyCfAteiYe12O2ybcRk1tTb25L6wfpdWl\n9AvVQsqmSf2c9TMTX5/ocpxp+HAI7FHG/l/d06T6h/U35j4fwx0h9Q/AMX5PqZVmaIKCygJ25tXL\n7tXpq5k8YDL+fv4M6z2MJXuWEB+uhVRD7769BXu5cNiFbMreRF5lNjMmaiFljzpRWehk2zf4HP9A\nm/petn7j6aJ9KSY0hvmnzrdH4ret+WUXUiFaSH2771sqD1eyLmOdy7r6xJWxeqUWUr1DepNbntvk\nccVVxcZxwgdxR0iJcojfY5n5zCRgFxRWFrIzt7GQAh394as9XxHfM57gbsFOmlRxVTHlNeWcOeRM\nfspcT6VfLufO6GMXUkopCioK6NndzaB9ho7ICUqp3ymlllu/ucAJ3m6Ut7hj0h32cF024WQz99k0\nqWX7lzEiegTfpXznsp6wqDJWr+hBeTlU58WxOyujyeNKquvHpMwS8r6DO0Jqn4jcbs3tCBCRO4B9\nnm6Yr1JQUcCuvF32TrA6YzUnxp0I6ICn2aXZJPRKaGTu21ewj2MijmF8v/H8kPoDfjW9GBgfQI/A\nHpRVl1FxuAJ/P/9Wr41k6FDUOk7dsCJO1HqxPR2GyOBIAv0Dncx9aUVprM1Yy59O+lOzQqqGMoYN\n6sHAgfDJmwNYtiG90TGH6w5TU1tDWGAYQf5Bzbq3GzoW7gipG9GDvZno6MsTgeubLdGFKagsoKq2\nisySTMqqy9iVt4vj+h8HaE0K9Kq0wQHBTt59e/P3MihiEAPCByC1IfTy1+YPmyZlxqM6BfcC34nI\nChFZCSwH7vZymzoEIkLf0L5OjhNf7/2aUX1GcdaQs/j5wM+NvGFtlNWU8diDoXzyCfzxtjj2H2os\npEqqSggNDEVECAsKMx5+PkSLZjul1EF0rDBDE3zx6xdkFmdyw/E3oJSisLKQ8f3GszN3JwH+AYyJ\nGWPXfkb1GQXQpOPE3gItpLKzhcNp4xhuLcRghFTnQSm1TESGUL90zW466XI0beHjSz5mXN9xgNak\nSqtLmTlwJmFBYYyOGc3q9NXMPGZmo3Jl1WWMG9WDvqEQlDKAR1anU1ICYWH1x5RUlxAWpBPCAsMo\nqSqxj4MZOjbuzJPqLiK3iMjLIvKm7dcejeuIFFUW8X3q9/b9DQc28GP6j4AOcyQI4/qOY2feTqfx\nKNDC6fyh5xMXHufS3PfQQzCu73gG9rE0qQAjpDoTSqkqpdQW61cFfOztNnUUJsROsC8h06dHHwBm\nDtRC6ZTEU5xMfmsz1vLaz3pZLNtkXoChfQcgvdJZudJ5zKm4qpiwQC2kwoPCjfOED+GOue9doC96\nYcKV6GUCuqyu/MzqZ3jg2wfs+/kV+eSV5wGW911wBMN6D2NnrhZStvEo0CaN//72vwT6BxLcLdjJ\nu29vwV78igbx5Zfwt+uuZt74eYDWpMpqyoxnX+fFBGJsguiQaMb1HceJA3T/aSikPtj6Ae9ueRel\nFOU15YQE6JWqe3bvSYC/H18u08E8UgpTUEpRUlVCeFA4gDH3+RjuCKnBSqmHgTKl1NvokC4dc01u\nD1NeU87LG17mUEX91Jb8iny7y2tBRQG9uvdieO/h7MjbwZqMNfZO1pBG5r78vaz+YhA33QQTBg63\nRy7vEdiD0upSe92GTodxM2uCAP8ANt6w0W4qnxo/lS05Wyiq1MLnh7Qf2J67nYrDFQT6Bzot4tk/\nNI5v1ulxqRlvz2Btxlonc5/RpHwLd1zJrYD4FIrIKHT8vj6ea1LH5a1f3mJE9Ah25O6wpx2qOOSs\nSXWPYHj0cFanryYuPM5p+XdHHIVUTW0NmSWZfL0wgUd+cD4uNDCU7NJsY+7zYUTkc5oWRgJEtXNz\nfJLggGBOjNMLKJ6SeArJ+cn4iz/7C/bbTX02BvcZwJrKdH7ZHcv+wv3szNtJz6CednOfbUzK4Bu4\nI6RetdaTegj4HxAKPOzuCUTkDOB5tNb2hlJqfoP8cOA99Mqj/sACpdRb7pRtT2rranl2zbO8ef6b\nzHh7BrV1tfj7+WtNqszSpCq1thMbFktwt2Cn8aiGBAcEOy2h3cu/H8ckBjJ4sPNxxnGiU/BMG/MM\nDpw+6HS+2fsNgf6BTIidQNXhKtYfWG+P22cjvucAgs5PZ+rFwXAxbM3aRVK/4UaT8lGaFVIi4gcU\nK6UKgO+BY1pTuVX+JWAmcAC9LtVnSqldDofdAmxXSp0nIr2B3SLyHlDnRtl2Y0fuDvzEj2nx0wgP\nCqewspCokCjyK/IpqymjoqbCPiYlIgzrPYzJca6FlKMmlVaUBoUJXHFF4+Mcx6Sie0R76vIMHkQp\ntbLlowwtMWvQLC746ALCg8KZFj+NAyUHWJ+5vpEmNSB8AP2mZXDciWU8uSKWb3/ZzcCoOMIDrTGp\nQDMm5Us0OyZlRZe47wjqnwAkK6VSrYCaH9I4oKYCbM6iYcAhpdRhN8t6jNq6WmZ/Ops6VQdoU16f\nHn0QEXqH9Lab+A6VHyK4WzCHKg5RUFFgd26Yf+p8Lh15qcv6QwJCKCqr4NRT4Z9vF5B/IJLLLmt8\nXI+AHkaTMhjQUzgqaip4f+v7nJRwEiOiRzSpScWFx5FenM6+sl84b9Cl/HpoN8VVDi7oQcbc50u4\n4zjxrYjcIyIDRCTS9nOz/ljAcWZdhpXmyEvACBE5AGwG7mhFWY/x66Ff+WDbB5RV6+XbS6rrvYOi\nQqI4VHGI2rpaiquKGRQ5iNyyXCdBcsrAU5rVfIK7BZOVV05wsBaAIwb2IrqJw+3mvirj3Wfo2ogI\nswbNIqs0i0lxkxgRPYLNOZsba1I9B5BenM6m7E3cedol1PRIYcuv+cYF3UdxZ0zK9n1/i0OaopWm\nv2Y4HdiklJphhYxZKiJJra3k0UcftW9Pnz6d6dOnH1GjNmZtBKCoqoiwoDCKq4rrhVRwFHnleRRV\nFREeFE5MjxjyyvMoqCwgNsw9ORrcLYT8knLuuw9+8i8gs6RpAeQYu89oUu3DihUrWLFiRbucS0S6\nWZYDgxucOfhMkvOTCQkIYUT0CKprqxtpUgPCB5B8KJm88jyO6z+eyID+rNi9mZOSLgCMua8trM1Y\nS3zPeJeOYJ7EnYgTA4+g/ky0Q4SNOCvNkbnAX61z7RWR/cAwN8vacRRSRwObkLJ9cZVUldi/xKJC\nojhUfoj8inwigyPtkZcLKwvtoY9aImVPCLVSwZQp8M0K16a80MBQyqrLqKmtMUKqnWj4kfPYY48d\nUX0iskopNdXaflcpdZVD9k/oaOgGN7hk5CXMGjQLgNiwWMICwxppUjZzX1JMEkHdghgfP4xvd68m\nZfccmGA0qbYw/8f5nDX4LOYdN6/dz92ikBKROU2lK6XecaP+9cBgEUkAstDhlS5vcEwqcCrwo4jE\nAMeiA9gWuVHWY2zK3oQg9ofZUZPqHdybQxWHOFR+iMjgSKJDou2aVESweya5z/8TTEBIOX5+2itw\naNTQJo+zzZPyEz8jpHwXx7dow68YM5m3FfiJn72PiQgjokc00qTCgsLoGdTTHmJpZMxQlqZ8xesv\nh6KVMkYAACAASURBVDGkHBJmGk2qteSV51FYWeiVc7szJnWCw28a8ChwnjuVK6VqgVuBb4DtwIdK\nqZ0icoOI2ILUPgFMFpEtwFLgPqVUvquybl/ZEaCUYlP2JkbHjHYSUraB16gQbe7Lr8gnKiRKa1Jl\nuW5PuM3IgP98FEKtv/bua84pwrigdwqam7BrJvMeASOjRzbSpECPS9mWARnaW38AvvBUGPfeC6oq\n7KhoUvctva/LOGDkluVSUFnglXO7Y+67zXFfRHqhPe3cQim1hPqAmra0Vxy2s9DjUm6V9SSLkxdz\ncsLJHCw7SI+AHgyOHFxv7qsuIS48DtBjUvsL9juZ+7Yd3OZW6KIbboCPP4Ybbwriyboaautqm9XA\nbEKqurbaCCnfpZeIXIj+KOwlIhdZ6QKYBcKOgFOPObVJreiiYRdx2jGnAditFMMGhjFmDGTuCz9i\n4XK47jDPrX2Oi0dczITYCUdUly9gG87wBm1ZvLAMOJJxqg7JipQVnP3+2Tx80sOMiRnDuH7j6BnU\n0x6GxcncF6LNffkV+UR2jyS6RzR5FS2b+w4dgg8+gLQ06NVLeOEvekJvc1pScLdgKg9XEhIQQoB/\nwNG/cEN7sJJ668NK4FyHvO8bH25wl8tHNz0C8Ngp9eOINk0qPCicceNg/64wSgJaL6Q2ZW1icORg\nwoLCSC1M5XDdYVILUzu9kDpcd5iCioKOK6QahHTxA0YACz3ZqPampKqEuZ/N5e9n/Z2Hv3uYC4Ze\nwPi+4ymqKnI29wU6m/sOVRyqd5xo4ILeFLt3w7Bh0Ms6xDah13F+VUNEhNDAULuANPge1gq8Bi/R\nL7QfEd0j6NW9F+PHw1ff9iF/ZL52hgoKa7kCixu/vJGbj7+Z3439Hcn5yQCkFqV6qtkdhvyKfBSq\n45r7cA7bchhIVUo1vT6zj7Fw+0IW7VrEvoJ9zEicwc0n3My+gn0sWLOARZctYmPWRidzn6MLuk2T\nGhw5mOiQaHLKciitLm1WmOzapYWUjeBuwfZIFc0Jtx6BPYypr5MiIuOVUhu93Y7OjIiw/ebt9Avr\nx/jxMH9+CJNOn8Sy/cu4YNgFbtVRW1fL1pytbD24FYDkQ8l08+tGamHnF1KOsUm9gTuOE2nAOqXU\nSqXUj8AhEUn0aKvagTpVx93f3M2UAVO4+8S7+dtZfwPgwWkPckzEMZzQ/wQnV9WGjhMNXdD3Fewj\nPCgcP3F9S3fvhqEOI2x2TaoFM2FoYKgRUp2Xm7zdgK6AbcXf4cMhNRVOSzibL3/90u3ye/L3UHG4\ngm0Ht9n3T4w7sUtoUrlluUQGR3ZoIfUxOo6ejVo6wUJtazPWEh4Uzq0TbuXSkZfa16OJDI5kz217\niA2PdRJSjuvR2DSpvPI8u5Cqrq1u0WmioSYVEhBCfkU+tXW1BHcLdlnOCKnOi1Kq/SeedGECAmDk\nSIivOovFexajlHvOlVtytjC279h6TSo/mZkDZ3YNIVWey5DIIRRUeMfc546Q6qaUqrbtWNuBnmtS\n+/DRto+4bGQTwfLQ5gGAnkE9Ka5uPE8qqFsQ3bt1J6UwhcjgSAL8A+gZ1LNFQdJQkwoOCCarNMse\nlNYVRkj5NiKSICI9HfZPEZEXROT3IuLzfcnXGD8e8nYfS0hACJtzNrtVZnPOZs499lxKqkrIr8gn\nOT+ZU485tcuY+4ZEDenQmlSuiNjnRYnI+UCe55rkeWrravl4x8cuhZSN8KBwJ+8+m+MEaG1qX8E+\nooL1ckDRPaKbNdnV1EBKCk5LcYQEhJBZnNmiBtYjwIxJ+TgLsSb0ishYtCUiDRgDvOzFdnVJxo2D\njRvh7CHum/w252xmbN+xjOwzkl+yfyGtKI3j+x/P4brD9nfE0aa0upRBLw5yW9vzFLllucSHx6NQ\nVB6ubPfzuyOkbgQeFJE0EUkD7gdu8GyzPMuqtFX06dHH7prqCpu5TynltLIn6HGpmroaIoN1rN3e\nIb2bFST79kFsLHTvXp8WEhDCgZIDLQqg/2/vzMObrLIG/jstFFqWUmhZWqRQKaLIIgiIKBSpgLLo\np58KI26D67jggrsz4+g8M+PMuCHzueICyIiDIriyCFVwVECgRaBQbNkKVLbKZlnK/f64SZruSZo0\naXp+z5Onee/73puTNDfnPeeee07TqKaaXLZuE22M2el4Ph540xjzLDYlWHjHL4cgvXvDDz9YJTVj\n7Qx2HtpZbZ/M3Zn0aNOD7q2788mmT0hslkijBo1IbpEcMJffpn2byD2Qy/5f9wdkfE/Zc3QPCU0S\naNG4RVCsqWqVlDHmJ2PMedjQ87OMMecbYzYHXjT/s/foXp5Y/ATXfngtN/WqPirYqaSKThbRIKIB\nUZElnpn4mHgAl4JJiEmoUpE4w8/diW4Qzc7DO6tNpaTuvjqPuy/3IuBLcJXCUWqZnj2hqAjyFg9l\nbLexnPPqOcxeP7vS6w/8eoADRQdIiUvh7NZnMyd7Dp1bWpdIcmxywFx+m/fbn9n8Q5WmLAWsxRUo\naw7s72Z8TDxxjeOCsi5VrZISkb+ISAtjzGFjzGERiRORP9eGcP7mH9/8g8yCTD79zadMPG9itdc7\nlVRZVx9Yd1+Lxi2IjIgEcP0TKyM7u/R6FHhuSY05YwyDOw6uVl4lZFksIu+LyItAHLAYQETaAcer\n7Kn4nUaNYN48eOLxCC6K/COfjPuESQsm8dRXT1XoWssqyKJ76+5ESATdW3dnS+EWUlumAg4lFSBL\nKmef3YuVf7BqJTVh3gQeX/x4QGQAhyUVE8KWFHCJMcYlmaNK76WBEylwLN+5nLv63kXPtj09uj62\ncSwHjx0stUfKSavoVq71KIAOsR1cYa4VUZEl5VRS1bnyrjjzCs5NPNcjmZWQ5F7gQ2ALcIGjiCdA\nWyBwvy5KpZxxBrzxBtx+O/RN6st3N3/HpzmfMmnBpHLXZhZYVx/YwotAiZJqEThLKmd/DlGRUVVa\nUrkHcpmzYQ7L85cHRAawllRV7r6Tp05yw0c3eOQ29QVPlFSkiDRyHohINNCoiutDkuJTxfyw8wf6\nJvX1uI+z7swvRb+U25keHxPvWo8C+P2g3zOxf+XWWUWWVHSDaI8sKaVuYyzvGWOeN8bku7WvNsbM\nD6Zs9ZlLLoG8PDh6FNo2bcv88fOZsXYGawvWlrpucd5ierez1VQSmiTQpkmb0u4+N0tqS+EWbp53\nM/d+ca9X+7AqImd/Dv2T+rssqZ/2/8SNH91Y6prnvn2OO869gx9//pHjxYExyvcc2eNac68o68TK\nnSuZ9eMsRv97tKtIrD/xREm9C3wpIhNE5GZspvJ3/C5JgNm4byOtm7QupViqIzIi0lbQPbyrvCUV\n06rUWJERkS7XX1mOHIF16+Dss0u3xzSM4eCxgxoUUY8RkdeCLUN9pWFDe+O4bp09btG4Bb8f9Hse\nWPCAy+23KHcRmQWZjO8x3tXvL0P/wgUdLgAoFTiRfzCfodOGkhCTQHxMPLd8fEuNIvNy9uWQ1jHN\nZaEsz1/OO5nv8MPOHwBr4by79l0eueARTm95OlkFWT6/VmUYY1zuvrjGcRVaUotyF3Fn3zvp2aYn\nN3x0g99l8CRw4hlsOY0zsRnJ5wPJfpckwCzPX+6VFeWkeaPm7Di4o5yS6pbQjb6Jno03bx4MGACt\nWpVuj25oN/CqJVWvebX6S0BERohItohsEpGHKzh/hoj8V0SKROR+b/rWZ3r0gCy33/bb+tzG9oPb\nmbp6KoePH+bOz+5k8ojJrs3+AL8957euYKfk2GTyDuQx68dZDHlnCLf3uZ2/pv+Vxy98nAiJcAU/\neMsvRb9w9MRR+rTr43L3bdq3iVbRrZi8fDIAf1jyB64+62raNWtH38S+rNy50sdPoXIOHz9Mw4iG\nRDeMrtTdtyh3Eekp6fzfyP/jk02fcKL4RAUj+Y4nlhRAATbJ7FXY6KRaqevkT1bkr6BfovfRvk4l\nVTZwYkinITx90dOu4xMn7KMi3n0Xrr22fLvzi+9poUQl/DDG/FDdNSISAUzBlrTpBowTkTIrnOwD\n7gb+4UPfekuPHpDptp+3YWRDpo6ZytTVU0n4RwJntDqD0WeMrrR/u2btOHHqBK+vep2/DP0LDw58\nELAJAQZ3HMxXW7/ySa7N+zfTuWVn2jdv71JSOftzeOzCx5i3cR5TV03ls5zPeObiZwDom9iXFfkr\nfHotgA17NvD+uvJ5w/cc3VMqkrlsdN+R40dYuXMlFyZfSOMGjUlslsiWwi0+y1ERlSaYFZEu2Eq4\n47Cbd2cBYowZ4lcJaokVO1dUmta/KmIbx5J/KL+cJZWfD+vXw8W2ZA333AMffQS/+x3cfz80cdRh\n27MHli2D9yqowOVUUmpJhTciMq+q88aY6oqI9gNyjDFbHeO9B1wGZLuNsRfYKyKjvO1bn+nZEz7+\nuHTb+aedz7cTvmXXoV00jWpaZf8IiWDvg3srdPUPTh7M11u/5ubeN5c7V1hUyNzsuSzIXcDWwq0c\nKz7GKyNfoU9iH8AqpNRWqSQ1T3KtSW3at4k7+97JVWddxa2f3ErGDRmu346+SX15eeXLvnwEbC3c\nSvr0dGIaxnB1t6tLndtzxO6RAnszXdYyXLptKX0S+7g+p9RWqWzev5nUVqk+yVIRVVlS2ViraZQx\n5gJjzEvYvH11jmMnj/Hjzz+6ykl7Q2Xuvpkz4brr4Phxu/D63nswbRp8/jlMn15y3fvvw8iR0LSC\n77ozX5+uSYU9A4D2wFJsVYFnyzyqIwnY7na8w9HmCTXpG/Y43X0VLR21a9bOo1Iela1FD06u2JI6\neeok6dPSmb1hNkM6DuGvQ//KNd2u4e7P73atYeXsy6FzXGdaN2lNYVEhx04ecymuJwY9wbtXvMuF\nyReWvI82PfjpwE8VBi4cOnaIk6dOVijjwWMHGTZjGJMGTKLgcAH7ju4rdd65RwrszXThsdLuvkW5\ni0jvlO467hzX2WcXZ2VUpaSuAHYBS0TkdREZSulNiXWGrIIsUlul0iSqfJnp6qjM3bd2LRw4ALNn\nw4cfwnnnWavqkktsUUMns2fDuEoMOLWk6g1tgceAs4EXgYuBvY7KAr75gxS/0Lo1REVZz4i/6dKq\nC8dOHivn/nr2v88SFx3HvLHzuLn3zVyYfCH3D7ifY8XHmLVuFlBiSUVIBG2atmHtz2sRhFbRregQ\n24GxZ48tNWZUZBTdErqxevfqUu27Du0iZXIKrf7eiitmXcGxk8dKnV+6dSlJzZK4b8B9nJt4Lit2\nlnYZOoMmoGJ3n3M9yknnlp1dtbb8RaXuPmPMR8BHItIE6x64F2gtIi8Dc4wxC/wqSQBZlLuIgacN\n9KlvZZZUVhY8+CBMmQLR0bYsPNjUR1+5/exs3GhdChWha1L1A2NMMfAF8IVjO8c4IENE/mSMmeLB\nEPlAB7fj9o42T/Cq75NPPul6npaWRlpamocvU3dxWlPt2/t3XBFhUPIgvt76NR1bdMQYw8qdK/nH\nf//BiltWlEoqHSERPD/8ea6bcx3dErqRsz+HW3rbBPlJzZJYkreELq26VJmIekD7ASz8aaEr8hDg\n7s/v5tbet3LfgPsY9NYgsgqySgWQZRVkufZg9kvqx/L85YzoPIKpq6byac6ntG3atpSScg+c2Hd0\nH7kHckvt4ezcsjMLcxe6jjMyMsjIyPDxE7RUW/TQGHMEmAnMFJE4bPDEw4BHSkpERgAvYK22qY5o\nQffzk4BrsYEZDbFRhPHGmEIR2QL8gi0VcsIY43XkgzGGN9e8yfT/mV79xRXQPKo5h48fLmX2nzhh\nlc/Spda1d/gwjHGsKrRvX3JXdvw47N0L7SrZ4+uM7ottFFvxBUrY4FBOI7EKqiMwGZjjYfcVQGcR\nScZ6N8Y6xqn05Xzt666k6gtOJXVpAFIUDE4ezJMZT/L2mrfZvH8zp8wpnhv+HJ3iOpW7dlDyICb2\nn8ilMy8l/2A+XVp1ASCpeRJLtiypdp3njr53MOitQTw48EGaRjVlzoY5rP15LTOumEHjBo3pm9SX\nzILM0krq5yxGpo4EoH9Sf95Y/QYAk5dP5tx25zItcxp/GPwHgHIh6Eu3LWXAaQNoGNnQ1eZck3JS\n9kbnT3/6k6cfnQtPKvO6cGSbeM3xqBa3yKKhwE5ghYjMNca4L/j+E0f1X8ei771uGS5OAWmO1/WJ\nr7d+TVRkFP2T+vvUP7axVSDultTGjdChAzRrBo89Bjt2lCSOTUoqUVL5+dC2LTSo5FOOaRhD80bN\nK/VpK+GBiEzDuvo+A/5kjPnRm/7GmGIRuQt7Y+i82dsgIrfZ0+Y1EWkDrASaAadEZCI21+bhivr6\n793VfXr2hM8+C8zYN/a6kZS4FBo1aET75u1JbZlapTV0/4D7mdh/Ipv2baJN0zaAtaTeWvMWD53/\nUJWv1TW+K0M6DeGVla8wqsso7vj0Dt6/6n0aN7A/Tj3b9GTN7jWl+mTuzuTRCx4FrCV16ye3klWQ\nRWFRIa+PeZ2/pf+t1FYZdyX11ZavGJxcOl1bpxad2PbLNk6eOkmDCK/US6X4Z5TK8TayaBzwb7dj\nwfMw+QqZunoqE86ZUOUXoyqcysldSa1da+++oMTN58RdSW3fbpVZZcQ0jNGgifrBeOAIMBG4x+27\nKFgl07yyjk6MMV9g9ym6t73q9rwAOM3TvkoJF1wADzxgPSQNG1Z/vTc0iWrCJamXeNUnMiKSMxPO\ndB0nNUvi8PHDHkXMPX7h4wyfMZwpy6fwTPozDEoe5DrXs01P5mSXGO9FJ4vIK8yja7zdkZDUPIlG\nkY146qunuLb7tURIhCuyD0qUlDEGEeGrrV8x5dLS3upGDRrRtmlbthZupVNcJ/Yc2eNStr5SIwXg\nAR5HFjnSLY0APnBrNsBCEVkhIl5XMC0sKmTexnlc1+M6b7u6cCon98CJrKwSJVWW2FgoLoZDh2wA\nxWkV/mxYusZ3ZfbVlWdfVsIDY0yEMaaZ49Hc7dHMEwWlBJZOnWydtwUhusqe1Nz+ZDrdf1XRo00P\nRqaOZGL/idzQq3T2h55te5JVkOWKIFy/Zz2pLVNLVXfol9SPDzZ8UOFvZsPIhjRu0JjDxw9TWFRI\nzv6cCnOKdm5pI/ymLJ9C+vT0cue9JdCWlDeMBpa5J7MFBhpjdolIAlZZbTDGLKuoc9kF3/Y92jN2\n9liu7X5tqbsBb6nIksrKglsqUZkiJdbU9u1VK6kIidDEsSGIPxZ7lbrF+PEwY4bdLhJqJDZLBEqS\n2lbHG2PeqLA9PiaeplFN2VK4hU5xncgqyHIlznXSP6k/237ZVsqSc8dpTa3ZvYb+Sf1LKTgnqS1T\nyd6bzQvfv8CuQ7vIP5jvUrS+EGgl5U1k0VhKu/owxuxy/N0jInOw7sNqlVTugVz6v9GfPwz6A3f1\nu8tn4QFiIh2WVCPPLCmwSmrHDqukzjqrRi+vBAF/LPYqdYurr4ZHH7UekGbVb42qVTq26EiH2A4e\n7dmqjp5tepJZkFmpkprQe0KVGTacSWa/2lp+PcpJ55adefH7Fzmt+Wn0TezLwtyF3NjrRp9lDrS7\nzxVZJCJRWEVUbve9iMQCg4G5bm0xItLU8bwJMAzwaMH5P+v+w9VnXc3d/e/2eS3Kyc48Gzhx/JBV\nVgcOQGEhdOxYeR9nhF917j5FUUKD+HgYNMhmjQk1UuJSyLw9s/oLPaBX215k7rZjVaSk4mPiOSuh\n8jvruOg4HljwAG+veZuhKUMrvKZzy87kFeYx6fxJDDt9GAt+qpkfNaBKyrE/xBlZtA54zxmVJCK3\nul16OTDfGPOrW1sbYJmIrAa+Az72dG/Wx5s+rvJuwBuy11jltPIbexezahV07w4RVXxy7u6+qgIn\nFEUJHcaPh3dCtL6Dvzb8Oy0pY0ypOlme8sjARxjbbSwLr1vI+aedX+E157Q7h5GpIxnVZRQXp1zM\nwtyFnKpBEWqpSSr5UEFEjPN97D26l9Mnn07BpAJX6GVNGHL5Vr7ucRY3Fxzh1VfhrrtsWPkTT1Te\n56WXbP2omTMhJ8fepSl1FxHBGFMns614g/s8qo8UFVkvyMqVVXtK6jLZe7NJezuNIZ2GsGzbMrbd\nu63G3qbq6DqlKzOvnEnvdr19mkuBdvfVOp/lfMbQTkP9oqCKi2HV0raM63gfX35pj2fPhmuuqbpf\nUpLdS3XsWPnyHIqihCaNG9sUZm+9FWxJAkdqy1TGnDGGwcmDy2W9CBQ1dfmFnZKat3Eeo7v4x9WX\nmQnt2zZi2o1/5uBBm10iMRFSqwmySUqC5cvtelQtfAcURfETEyZYJVVcJ1NpV09kRCSvjX6N28+9\nnbZN29bKa445YwzFp3z/QMNKSR0vPs6i3EWM7OJbHOnJMomCv/7aLqZGRMBFF8HDD1dvRYFVUocO\n6XqUotQ1evWChARYtCjYkoQP6SnpPD7ocZ/7h5WS2rRvE+2ataN1k9Ze9XviCWv1NG4Mb7htMXAq\nKYChQ+Hnn+Gqq6ofr21bq9g0sk9R6h4TJsCbbwZbCsVJKG3mrTF5B/Lo1KJ84saq2LvXZjL//ns4\ndQrS0uwO9MhIyMiAybZSM6NG2WKGKSnVj9mggVVUakkpSt1j3Dibk/PAAYjTrGVBJ7yUVKH3Smrm\nTKuAznBkNnvvPVsTqmNHeO65kvT97drBs56Up3OQlKSWlKLUReLiYPhw+1twxx3BlkYJK3df3oG8\nClPgV8Xbb8ONN5YcDxlis0WsX1+63VuuuQYGDPC9v6IowePGG+1vgxJ8wktJFeaREueBP85BZibs\n22eDItyJj696s64nPPCApkRSlLrKsGElN6tKcAk7JeWNu++11+D662uukBRFCS8iI+Haa+1ygBJc\nwubn2Rjjlbtv7Vr4z39sBglFUZSyjBoFn38ebCmUsFFS+3/dT2REpEc5rk6dgttvh6efhjY1q8el\nKEqYMmAA5ObC7t3BlqR+EzZKKvdAbqWuvg0bYO7ckuO337aKqrKaUIqiKA0bQno6zJ8fbEnqN2Gj\npPIKK3f1PfII3HSTzQJx4gQ89RQ8/7yuRSmKUjWXXKIuv2ATNj/TlW3k3bgRvvvOZo54+WVbfTM1\nFc47LwhCKopSpxgxAhYutLXhhg2DP/6xfPo0JbCEzWbevMI8urfuXq79uefshrwrr4SLL7ZVN9+o\nuLqyoihKKRIT7ab8Hj3gnnvg22/tDe+nn2o2itoirJTUmDPGlGrbs8dG8GVnQ+vWdiF0796SfHyK\noijV8fTTNq/nxRfbtezrr7fLBU89FWzJ6gdhU/QwdXIqc8fO5cyEM13tU6bYkhnTptnjPXtsjSdn\nqiNF8QQteqi4k5sL/frBTz9BbGywpalb1Ouih9t+2UbHFh1Ltc2dC//zPyXHCQmqoBRFqRkpKTag\n4l//CrYk9YOwUVKJzRKJbhjtOi4stJnNhw0LolCKooQljz4KL7xgAyqUwBJwJSUiI0QkW0Q2icjD\nFZyfJCKrRWSViKwVkZMi0sKTvu70aNOj1PHnn9u1pyZN/Pp2FEVROOssW86jTx94991gSxPeBFRJ\niUgEMAUYDnQDxolIV/drjDH/NMacY4zpDTwKZBhjCj3p605ZJTV3Llx2mV/fjqIoiot777Xh6ffc\nA3l5wZYmfAm0JdUPyDHGbDXGnADeA6pSHeOAf/vS111JHT9ud4mPHl1T8RVFUSqnVy97MzxvXrAl\nCV8CraSSgO1uxzscbeUQkWhgBPCBt30Berbp6Xq+ZAl07Wqr4yqKogSSyy+Hjz4KthThSyjtkxoN\nLDPGFPrSefrk6USI1bkrV6Zx5ZVpfhRNqU9kZGSQkZERbDGUOkJ6OowfD/v3Q8uWwZYm/AjoPikR\nOQ940hgzwnH8CGCMMc9UcO2HwPvGmPd86Ova31FcbHeJf/cddPKuSK+iVIjuk1Kq4/LLbVab664L\ntiShTSjuk1oBdBaRZBGJAsYC5by3IhILDAbmetu3LMuWQVKSKihFUWqPyy4rXWlB8R8BVVLGmGLg\nLmABsA54zxizQURuE5Fb3S69HJhvjPm1ur6VvVafPpCRAR98AFdcEYA3oyiKUgmjR9u18MzMYEsS\nfoRNWqSoKENsrI3s+/ZbOPPM6vspiieou0/xhH//2+6dWr7cZrdRyhOK7r5ao21b+PJLuOEGVVCK\notQ+48bZx+jRsGVLsKUJH8JGSTVvDt27w4svBlsSRVHqK3/+sw2iOPdceOedYEsTHoSNu2/gQMOy\nZcGWRAlH1N2neMu6dXDhhbboqrr+SqjX7r7mzYMtgaIEDk/yWIrIZBHJEZE1InKOW/sWEcl05Mhc\nXntS11+6dYOrr1bPjj9QJaUoIY4neSxF5BLgdGNMKnAb8LLb6VNAmiNHZr9aErve89BD8MorcPBg\nsCWp26iSUpTQx5M8lpcB0wCMMd8DsSLSxnFOCKO5XldISYHhw+06VXFxsKWpu4TNF1crZCphjCd5\nLMtek+92jQEWisgKEbklYFIq5Xj6afjmG1vaY/HiYEtTNwml3H01Qi0pRamUgcaYXSKSgFVWG4wx\nFYYZPfnkk67naWlppKWl1Y6EYUpKis2C8/77cNttkJ0NkZHBlqr28EcezLCJ7nvhBcPEicGWRAlH\ngh3d50keSxF5BVhijJnlOM4GBhtjCsqM9UfgkDHmuQpeR6P7AoQx0L+/3ex7+eXBliZ4aHSfooQn\nnuSxnAdcDy6lVmiMKRCRGBFp6mhvAgwDfqw90RUAEXjgAXj22WBLUvdQJaUoIY4nOTCNMZ8BeSKy\nGXgV+J2jextgmYisBr4DPjbGLKj1N6Fw5ZWwfbtNm6R4Tti4+xYuNKSnB1sSJRwJtruvtlB3X+B5\n7jlYvRqmTw+2JMFB3X2KoighzPXXw8cfw4EDwZak7qBKSlEUpZaIj4cRI2DGjGBLUndQJaUoilKL\n3HILvP66jfjbsQOKioItUWgTNkpKN/MqilIXGDIEjhyxCWjPPBMGDIDNm2HpUhg7FrZuDbaEBPkX\nfQAADJ9JREFUoUXYBE6cOmWQsF/aVoKBBk4o/ubzz6GgwCqlqVPh0UehVStbYfzkSfjoo2BLGBh8\nmUtho6TC4X0ooYkqKSXQ7NplldSpU7Yu3gsvwMiRwZbK/6iSUpQAoEpKqU3mz4c77oD166Fx42BL\n419CMgTdwzo4aY5aNz+KyBK3dq2DoyhKvWL4cFuP6pVXgi1JaBBQS8pRB2cTMBTYiU3vMtYYk+12\nTSzwX2CYMSZfROKNMXsd53KBPsaYKncV6B2gEkjUklJqm6wsGDYMcnKgWbNgS+M/fJlLgc6C7qqD\nAyAizjo42W7X/Ab4wBiTD+BUUA60Do6iKPWOHj0gPd3WoureHXbvhkmTgi1VcAi0AvCkDk4XoKWI\nLHHUu7nO7ZzWwVEUpV7y1FM28u/99+3zffuCLVFwCIV6Ug2A3sBFQBPgWxH51hizGS/q4CiKooQT\nKSmw1+FXGj0aFi2Ca64JrkzBINBKKh/o4Hbc3tHmzg5grzGmCCgSka+BnsBmY8wuAGPMHhGZg3Uf\narE2JaD4o1CboviTESPgiy+sktq61Sqs3/6WerE3NNCBE5HARmzgxC5gOTDOGLPB7ZquwEvACKAR\n8D1wDbAFiDDGHHbUwVkA/KmiMgO64KsEEg2cUILN5s0waBDk58NvfmPD1IcNs+7AJk2CLZ3nhFwI\nuod1cLKB+UAWtt7Na8aY9WgdHEVRFAA6d4aYGJg1CxYvho0boUEDa1mF+32FbuZVlGpQS0oJBe68\nE955B37/e3j4YTh+HHr2hL/9DS67LNjSeUbIWVKKoiiKf7j0UoiOtsoKICoKpkyBiRPh6NGS67Kz\nw6telVpSilINakkpoYAxNtovIaF0+9ix0L49/POfsH+/zayelGTdgi1aBEfWytDcfYoSAFRJKaHM\n3r02e/pLL9ns6U2b2qi/FStsRGAo1dpTJaUoAUCVlBLqfPutzZretCmsW2cj/u69FxYuhDlzoGvX\nYEtoUSWlKAFAlZRSF5g1CxITbTFFJ2++CY88Ai+/DFdeGTzZnKiSUpQAoEpKqcusXAn/+7/28bvf\nQadOwdsErNF9iqIoSinOPdcqqn37YOBAWwbkyJFgS+U5qqQURVHCnPh4eOst2LkTUlNh+vRgS+Q5\nqqQURVHqCSI2oGLy5LqTqUKVlKIoSj0iLQ0iI22S2urYsaMkE3uwUCWlKIpSjxCxWSr+/nc4ccK2\nHTxoqwC7s3+/TWo7fDj8+mvty+lEo/sUpRo0uk8JN379FS6/HPbssVnVn30WTp2CtWuhdWsoLoZR\no2z2il27oFEjuOUWeOUVW+fqppugY0fvX1ej+xRFUZRqiY622SjuucduBP74Y7jhBmthHTsGt94K\nRUXW2nrjDVi1yp7v3dvmBezTB2bOrB1Z1ZJSlGpQS0qpDxw9arOqR0ZaC2raNGjWzJ47dsyWBomM\ntMdr18LQoTabxcCBJWNs327XsJo3h9NPL/8auplXUQKAKimlvrBiBXzzjbWwIqrxs82fb62rxx6z\n61ZPPmnTMLVvbwMuXn21fJYLVVKKEgBUSSlKxXz/PTz/PHzyiVVsTzxhizOuXm1Li0ycCAMGQJcu\n0K6dKqlgi6GEKaqkFKVqjCmfamn9enjmGcjNtVkvnn9elVSwxVDCFFVSiuIfNLpPURRFCStUSSmK\noighS8CVlIiMEJFsEdkkIg9Xck2aiKwWkR9FZIk3ff1NRkaGjqVjhRwezqPJIpIjImtEpJc3ff1N\nKP8vQ1W2+jCWLwRUSYlIBDAFGA50A8aJSNcy18QC/wJGGWPOBq7ytG8gCNV/ro4VvLGCjYfz6BLg\ndGNMKnAb8IqnfQNBKP8vQ1W2+jCWLwTakuoH5BhjthpjTgDvAZeVueY3wAfGmHwAY8xeL/oqSn3A\nk7lwGTANwBjzPRArIm087KsoIUuglVQSsN3teIejzZ0uQEsRWSIiK0TkOi/6Kkp9wJO5UNk1Oo+U\nuo0xJmAP4ErgNbfj8cDkMte8BPwXaAy0AjYBnT3p63bO6EMfgXwEcp74aR59DJzvdrwI6O1JX51H\n+qjNh7ff/wYElnygg9txe0ebOzuAvcaYIqBIRL4GenrYF6Be7GFR6jWezIV84LQKronyoC+g80gJ\nTQLt7lsBdBaRZBGJAsYC88pcMxe4QEQiRSQG6A9s8LCvotQHPJkL84DrAUTkPKDQGFPgYV9FCVkC\nakkZY4pF5C5gAVYhTjXGbBCR2+xp85oxJltE5gNZQDHWNbEeoKK+gZRXUUIRD+fRZyJyqYhsBo4A\nN1XVN0hvRVG8JizSIimKoijhSZ3OOFGTTYoiMlVECkQky60tTkQWiMhGEZnv2MPlyVjtRWSxiKwT\nkbUico+v44lIIxH53rG5ea2I/LEmsjn6RojIKhGZ54extohIpkO+5TUZT0RiReQ/IrLB8dn19/Ez\n6+KQZ5Xj7y8ick8N5LrPsbE8S0TeFZGomnxmdQGdSx6/V7/MJZ1HXowVzKilGkY8RQCbgWSgIbAG\n6OpF/wuAXkCWW9szwEOO5w8Df/NwrLZAL8fzpsBGoGsNxotx/I0EvsPudfFpLMf19wEzgHk1eZ+O\n63OBuDJtvr7Pt4GbHM8bALE1kc3te7ETG0Tg9VhAouM9RjmOZwE31FSuUH7oXKr9uaTzyIuxanMy\n+PMBnAd87nb8CPCwl2Mkl5lY2UAbx/O2QLaPsn0EpNd0PCAGWAn09XUsbDTXQiDNbWL5LBeQB7Qq\n0+b1eEBz4KcK2mv6mQ0DltZArkRgKxDnmPDz/PG/DOWHzqXan0s6jzwfqy67+wKxSbG1sRFRGGN2\nA629HUBEOmLvKr/D/jO8Hs/hUlgN7AYWGmNW+DoW8DzwIHaPghNfx8IxzkKxG69vrsF4nYC9IvKW\nw73wmtjozprIBnANMNNXuYwxO4FngW3YUO1fjDGL/CBXKKNzyTP8OZd0Hnk4Vl1WUrWBV1ElItIU\nmA1MNMYcrqC/R+MZY04ZY87B3rn1E5FuvowlIiOBAmPMGqCqPTDevM+BxpjewKXAnSJyoS+yYe+u\negP/cox3BHsH79NnBiAiDYExwH8q6evJZ9YCmzYoGXs32ERErq2JXAqgc6ksOo88HKsuKymPN/t6\nQYHYfGeISFvgZ087ikgD7KSaboyZW9PxAIwxB4EMYISPYw0ExohILvBv4CIRmQ7s9lUuY8wux989\nWFdMPx9l2wFsN8asdBx/gJ1sNfnMLgF+MCX5H30ZKx3INcbsN8YUA3OA82soV6ijc6l6/DqXdB55\nPlZdVlL+2KQolL4rmgfc6Hh+A3ajsae8Caw3xrxYk/FEJN4Z8SIi0cDF2M3NXo9ljHnMGNPBGJOC\n/XwWG2Ouw6bQ8WoshzwxjjtcRKQJ1m+91kfZCoDtItLF0TQUWOfLWG6Mw/6AOPFlrG3AeSLSWETE\nIdf6GsoV6uhcqgZ/ziWdR16O5c1iWqg9sHdFG4Ec4BEv+87ERq8cc3ygN2EX+RY5xlwAtPBwrIHY\njchrgNXAKodsLb0dD+ju6L8Gu8H5cUe712OVGXcwJYu9Po2F9X873+Na52deg/F6Yn8g1wAfYqOS\nfB0rBtgDNHNr83WsP2J/zLKAd7ARbzX6/EP9oXOp9uaSziPvxtLNvIqiKErIUpfdfYqiKEqYo0pK\nURRFCVlUSSmKoighiyopRVEUJWRRJaUoiqKELKqkFEVRlJBFlVQdRkSK3dLqrxKRh/w4drKIrPXX\neIoSqug8Cm0CWplXCThHjM3XFSh0E51SH9B5FMKoJVW3qTDRpYjkicgzjmJj34lIiqM9WUS+FJE1\nIrJQRNo72luLyIeO9tUicp5jqAaOrMo/isgXItKolt6XotQmOo9CGFVSdZvoMm6Kq9zOHTDG9AD+\nBThzoL0EvGWM6YVNZfOSo30ykOFo743N/QWQCrxkjDkb+AW4MsDvR1GCgc6jEEbTItVhROSgMaZ5\nBe15wBBjzBZHRuldxpgEEdkDtDXGFDvadxpjWovIz0CSMeaE2xjJwAJjzBmO44eABsaYv9TKm1OU\nWkLnUWijllT4Yip57g3H3J4Xo2uYSv1D51GQUSVVt6mq+No1jr9jgW8dz7/BpuEHGA8sdTxfBPwO\nXJVMnXeVVY2vKOGCzqMQRjV63aaxiKzCTgIDfGGMecxxLk5EMoEiSibUPcBbIjIJm47/Jkf7vcBr\nIjIBOAncgS23rb5gpT6g8yiE0TWpMMThS+9jjNkfbFkUpa6i8yg0UHdfeKJ3HopSc3QehQBqSSmK\noighi1pSiqIoSsiiSkpRFEUJWVRJKYqiKCGLKilFURQlZFElpSiKooQs/w/WQ4ikQPvW9wAAAABJ\nRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fdc19715510>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_performance(history_finetune)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Discussion**\n",
    "\n",
    "We get a test accuracy of ~89%. This is almost a 5% jump over the training from scratch case. This is a huge jump in test accuracy! One should not expect such a bug jump usually. We should bear in mind that the ImageNet dataset, on which AlexNet was pre-trained already contained dogs and cats among its classes. However, many papers report benefits of fine-tuning over training from scratch."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Task 3 : Using AlexNet as a feature extractor\n",
    "\n",
    "We treat the activation maps from outout of the last convolutional layer as dense feature maps. These are flattened and fed into a single layer ANN consisting of 256 neurons.\n",
    "\n",
    "In the cell below, we get a sliced CNN model (alexnet_convolutional_only) which will enable us to extract features from any layer specified by its name (\"convpool_5\" in our case)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "alexnet = get_alexnet(input_size,nb_classes,mean_flag)\n",
    "alexnet.load_weights('../convnets-keras/weights/alexnet_weights.h5', by_name=True)\n",
    "\n",
    "alexnet_convolutional_only = Model(input=alexnet.input, output=alexnet.get_layer('convpool_5').output)    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get the output of the sliced CNN model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found 2000 images belonging to 2 classes.\n",
      "Found 800 images belonging to 2 classes.\n"
     ]
    }
   ],
   "source": [
    "#code ported from https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html\n",
    "\n",
    "import numpy as np\n",
    "\n",
    "generator = train_datagen.flow_from_directory(\n",
    "        '../Data/Train',\n",
    "        target_size=(227, 227),\n",
    "        batch_size=batch_size,\n",
    "        class_mode=None,  # this means our generator will only yield batches of data, no labels\n",
    "        shuffle=False)  # our data will be in order, so all first 1000 images will be cats, then 1000 dogs\n",
    "# the predict_generator method returns the output of a model, given\n",
    "# a generator that yields batches of numpy data\n",
    "train_data = alexnet_convolutional_only.predict_generator(generator, 2000)\n",
    "train_labels = np.array([[1, 0]] * 1000 + [[0, 1]] * 1000)\n",
    "\n",
    "generator = test_datagen.flow_from_directory(\n",
    "        '../Data/Test',\n",
    "        target_size=(227, 227),\n",
    "        batch_size=batch_size,\n",
    "        class_mode=None,\n",
    "        shuffle=False)\n",
    "\n",
    "validation_data = alexnet_convolutional_only.predict_generator(generator, 800)\n",
    "validation_labels = np.array([[1,0]] * 400 + [[0,1]] * 400)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We define and train a single layer ANN model with 256 neurons"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 2000 samples, validate on 800 samples\n",
      "Epoch 1/80\n",
      "0s - loss: 0.2403 - acc: 0.6190 - val_loss: 0.1720 - val_acc: 0.7625\n",
      "Epoch 2/80\n",
      "0s - loss: 0.1821 - acc: 0.7230 - val_loss: 0.1565 - val_acc: 0.7925\n",
      "Epoch 3/80\n",
      "0s - loss: 0.1664 - acc: 0.7550 - val_loss: 0.1502 - val_acc: 0.8000\n",
      "Epoch 4/80\n",
      "0s - loss: 0.1528 - acc: 0.7830 - val_loss: 0.1508 - val_acc: 0.7987\n",
      "Epoch 5/80\n",
      "0s - loss: 0.1464 - acc: 0.7950 - val_loss: 0.1425 - val_acc: 0.8063\n",
      "Epoch 6/80\n",
      "0s - loss: 0.1350 - acc: 0.8210 - val_loss: 0.1418 - val_acc: 0.8025\n",
      "Epoch 7/80\n",
      "0s - loss: 0.1263 - acc: 0.8325 - val_loss: 0.1373 - val_acc: 0.8125\n",
      "Epoch 8/80\n",
      "0s - loss: 0.1181 - acc: 0.8425 - val_loss: 0.1354 - val_acc: 0.8125\n",
      "Epoch 9/80\n",
      "0s - loss: 0.1093 - acc: 0.8550 - val_loss: 0.1371 - val_acc: 0.8187\n",
      "Epoch 10/80\n",
      "0s - loss: 0.1027 - acc: 0.8670 - val_loss: 0.1472 - val_acc: 0.7887\n",
      "Epoch 11/80\n",
      "0s - loss: 0.0969 - acc: 0.8755 - val_loss: 0.1527 - val_acc: 0.7762\n",
      "Epoch 12/80\n",
      "0s - loss: 0.0923 - acc: 0.8795 - val_loss: 0.1466 - val_acc: 0.7900\n",
      "Epoch 13/80\n",
      "0s - loss: 0.0858 - acc: 0.8980 - val_loss: 0.1318 - val_acc: 0.8150\n",
      "Epoch 14/80\n",
      "0s - loss: 0.0792 - acc: 0.9005 - val_loss: 0.1308 - val_acc: 0.8250\n",
      "Epoch 15/80\n",
      "0s - loss: 0.0791 - acc: 0.9085 - val_loss: 0.1290 - val_acc: 0.8250\n",
      "Epoch 16/80\n",
      "0s - loss: 0.0727 - acc: 0.9145 - val_loss: 0.1323 - val_acc: 0.8213\n",
      "Epoch 17/80\n",
      "0s - loss: 0.0678 - acc: 0.9260 - val_loss: 0.1286 - val_acc: 0.8213\n",
      "Epoch 18/80\n",
      "0s - loss: 0.0680 - acc: 0.9220 - val_loss: 0.1284 - val_acc: 0.8237\n",
      "Epoch 19/80\n",
      "0s - loss: 0.0625 - acc: 0.9335 - val_loss: 0.1313 - val_acc: 0.8287\n",
      "Epoch 20/80\n",
      "0s - loss: 0.0602 - acc: 0.9375 - val_loss: 0.1310 - val_acc: 0.8137\n",
      "Epoch 21/80\n",
      "0s - loss: 0.0547 - acc: 0.9415 - val_loss: 0.1334 - val_acc: 0.8200\n",
      "Epoch 22/80\n",
      "0s - loss: 0.0540 - acc: 0.9465 - val_loss: 0.1258 - val_acc: 0.8250\n",
      "Epoch 23/80\n",
      "0s - loss: 0.0525 - acc: 0.9445 - val_loss: 0.1280 - val_acc: 0.8125\n",
      "Epoch 24/80\n",
      "0s - loss: 0.0493 - acc: 0.9495 - val_loss: 0.1356 - val_acc: 0.8163\n",
      "Epoch 25/80\n",
      "0s - loss: 0.0501 - acc: 0.9490 - val_loss: 0.1295 - val_acc: 0.8225\n",
      "Epoch 26/80\n",
      "0s - loss: 0.0432 - acc: 0.9585 - val_loss: 0.1287 - val_acc: 0.8187\n",
      "Epoch 27/80\n",
      "0s - loss: 0.0413 - acc: 0.9615 - val_loss: 0.1314 - val_acc: 0.8200\n",
      "Epoch 28/80\n",
      "0s - loss: 0.0377 - acc: 0.9665 - val_loss: 0.1273 - val_acc: 0.8213\n",
      "Epoch 29/80\n",
      "0s - loss: 0.0378 - acc: 0.9685 - val_loss: 0.1262 - val_acc: 0.8187\n",
      "Epoch 30/80\n",
      "0s - loss: 0.0350 - acc: 0.9680 - val_loss: 0.1263 - val_acc: 0.8225\n",
      "Epoch 31/80\n",
      "0s - loss: 0.0328 - acc: 0.9720 - val_loss: 0.1253 - val_acc: 0.8237\n",
      "Epoch 32/80\n",
      "0s - loss: 0.0312 - acc: 0.9700 - val_loss: 0.1286 - val_acc: 0.8163\n",
      "Epoch 33/80\n",
      "0s - loss: 0.0320 - acc: 0.9710 - val_loss: 0.1305 - val_acc: 0.8200\n",
      "Epoch 34/80\n",
      "0s - loss: 0.0295 - acc: 0.9740 - val_loss: 0.1287 - val_acc: 0.8213\n",
      "Epoch 35/80\n",
      "0s - loss: 0.0277 - acc: 0.9800 - val_loss: 0.1257 - val_acc: 0.8250\n",
      "Epoch 36/80\n",
      "0s - loss: 0.0253 - acc: 0.9810 - val_loss: 0.1270 - val_acc: 0.8175\n",
      "Epoch 37/80\n",
      "0s - loss: 0.0239 - acc: 0.9815 - val_loss: 0.1294 - val_acc: 0.8150\n",
      "Epoch 38/80\n",
      "0s - loss: 0.0237 - acc: 0.9810 - val_loss: 0.1289 - val_acc: 0.8175\n",
      "Epoch 39/80\n",
      "0s - loss: 0.0226 - acc: 0.9840 - val_loss: 0.1285 - val_acc: 0.8337\n",
      "Epoch 40/80\n",
      "0s - loss: 0.0222 - acc: 0.9840 - val_loss: 0.1316 - val_acc: 0.8187\n",
      "Epoch 41/80\n",
      "0s - loss: 0.0227 - acc: 0.9785 - val_loss: 0.1245 - val_acc: 0.8313\n",
      "Epoch 42/80\n",
      "0s - loss: 0.0201 - acc: 0.9875 - val_loss: 0.1259 - val_acc: 0.8300\n",
      "Epoch 43/80\n",
      "0s - loss: 0.0212 - acc: 0.9825 - val_loss: 0.1279 - val_acc: 0.8213\n",
      "Epoch 44/80\n",
      "0s - loss: 0.0203 - acc: 0.9845 - val_loss: 0.1281 - val_acc: 0.8275\n",
      "Epoch 45/80\n",
      "0s - loss: 0.0175 - acc: 0.9900 - val_loss: 0.1262 - val_acc: 0.8287\n",
      "Epoch 46/80\n",
      "0s - loss: 0.0171 - acc: 0.9905 - val_loss: 0.1287 - val_acc: 0.8263\n",
      "Epoch 47/80\n",
      "0s - loss: 0.0169 - acc: 0.9885 - val_loss: 0.1265 - val_acc: 0.8313\n",
      "Epoch 48/80\n",
      "0s - loss: 0.0161 - acc: 0.9885 - val_loss: 0.1306 - val_acc: 0.8163\n",
      "Epoch 49/80\n",
      "0s - loss: 0.0140 - acc: 0.9915 - val_loss: 0.1276 - val_acc: 0.8300\n",
      "Epoch 50/80\n",
      "0s - loss: 0.0142 - acc: 0.9915 - val_loss: 0.1260 - val_acc: 0.8263\n",
      "Epoch 51/80\n",
      "0s - loss: 0.0153 - acc: 0.9890 - val_loss: 0.1243 - val_acc: 0.8363\n",
      "Epoch 52/80\n",
      "0s - loss: 0.0138 - acc: 0.9925 - val_loss: 0.1251 - val_acc: 0.8325\n",
      "Epoch 53/80\n",
      "0s - loss: 0.0131 - acc: 0.9920 - val_loss: 0.1287 - val_acc: 0.8263\n",
      "Epoch 54/80\n",
      "0s - loss: 0.0125 - acc: 0.9920 - val_loss: 0.1256 - val_acc: 0.8375\n",
      "Epoch 55/80\n",
      "0s - loss: 0.0133 - acc: 0.9895 - val_loss: 0.1251 - val_acc: 0.8275\n",
      "Epoch 56/80\n",
      "0s - loss: 0.0118 - acc: 0.9925 - val_loss: 0.1272 - val_acc: 0.8325\n",
      "Epoch 57/80\n",
      "0s - loss: 0.0109 - acc: 0.9950 - val_loss: 0.1271 - val_acc: 0.8287\n",
      "Epoch 58/80\n",
      "0s - loss: 0.0109 - acc: 0.9945 - val_loss: 0.1288 - val_acc: 0.8337\n",
      "Epoch 59/80\n",
      "0s - loss: 0.0116 - acc: 0.9920 - val_loss: 0.1265 - val_acc: 0.8287\n",
      "Epoch 60/80\n",
      "0s - loss: 0.0107 - acc: 0.9940 - val_loss: 0.1260 - val_acc: 0.8287\n",
      "Epoch 61/80\n",
      "0s - loss: 0.0097 - acc: 0.9940 - val_loss: 0.1265 - val_acc: 0.8287\n",
      "Epoch 62/80\n",
      "0s - loss: 0.0093 - acc: 0.9970 - val_loss: 0.1272 - val_acc: 0.8350\n",
      "Epoch 63/80\n",
      "0s - loss: 0.0094 - acc: 0.9940 - val_loss: 0.1256 - val_acc: 0.8275\n",
      "Epoch 64/80\n",
      "0s - loss: 0.0096 - acc: 0.9930 - val_loss: 0.1250 - val_acc: 0.8325\n",
      "Epoch 65/80\n",
      "0s - loss: 0.0079 - acc: 0.9960 - val_loss: 0.1265 - val_acc: 0.8300\n",
      "Epoch 66/80\n",
      "0s - loss: 0.0069 - acc: 0.9980 - val_loss: 0.1277 - val_acc: 0.8325\n",
      "Epoch 67/80\n",
      "0s - loss: 0.0068 - acc: 0.9980 - val_loss: 0.1269 - val_acc: 0.8275\n",
      "Epoch 68/80\n",
      "0s - loss: 0.0071 - acc: 0.9970 - val_loss: 0.1282 - val_acc: 0.8287\n",
      "Epoch 69/80\n",
      "0s - loss: 0.0081 - acc: 0.9965 - val_loss: 0.1234 - val_acc: 0.8413\n",
      "Epoch 70/80\n",
      "0s - loss: 0.0081 - acc: 0.9950 - val_loss: 0.1252 - val_acc: 0.8300\n",
      "Epoch 71/80\n",
      "0s - loss: 0.0071 - acc: 0.9970 - val_loss: 0.1286 - val_acc: 0.8275\n",
      "Epoch 72/80\n",
      "0s - loss: 0.0078 - acc: 0.9950 - val_loss: 0.1282 - val_acc: 0.8313\n",
      "Epoch 73/80\n",
      "0s - loss: 0.0052 - acc: 0.9990 - val_loss: 0.1297 - val_acc: 0.8213\n",
      "Epoch 74/80\n",
      "0s - loss: 0.0076 - acc: 0.9950 - val_loss: 0.1273 - val_acc: 0.8325\n",
      "Epoch 75/80\n",
      "0s - loss: 0.0058 - acc: 0.9975 - val_loss: 0.1269 - val_acc: 0.8325\n",
      "Epoch 76/80\n",
      "0s - loss: 0.0065 - acc: 0.9970 - val_loss: 0.1289 - val_acc: 0.8287\n",
      "Epoch 77/80\n",
      "0s - loss: 0.0051 - acc: 0.9985 - val_loss: 0.1271 - val_acc: 0.8263\n",
      "Epoch 78/80\n",
      "0s - loss: 0.0063 - acc: 0.9975 - val_loss: 0.1275 - val_acc: 0.8313\n",
      "Epoch 79/80\n",
      "0s - loss: 0.0051 - acc: 0.9985 - val_loss: 0.1290 - val_acc: 0.8287\n",
      "Epoch 80/80\n",
      "0s - loss: 0.0055 - acc: 0.9975 - val_loss: 0.1281 - val_acc: 0.8263\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "\n",
    "model = Sequential()\n",
    "model.add(Flatten(input_shape=train_data.shape[1:]))\n",
    "model.add(Dense(256, activation='relu'))\n",
    "model.add(Dropout(0.5))\n",
    "model.add(Dense(2, activation='softmax'))\n",
    "\n",
    "sgd = SGD(lr=0.001, decay=1e-6, momentum=0.9, nesterov=True)\n",
    "model.compile(optimizer=sgd,\n",
    "              loss='mse',\n",
    "              metrics=['accuracy'])\n",
    "\n",
    "history_convpool_5 = model.fit(train_data, train_labels,\n",
    "                      nb_epoch=80,\n",
    "                      batch_size=batch_size,\n",
    "                      validation_data=(validation_data, validation_labels),\n",
    "                      verbose=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAakAAAEbCAYAAABgLnslAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnXd4FFXXwH+H3iGhS+8gVV5ARJEAAgFBFBUBFcXuq9gL\nop+ABaVbEF8QEESkidKklyAIAtJ7DyWAtBBKICTZ8/0xk7AJm2QTdrOb5P6eZ57MrXNmM3fO3HPP\nvVdUFYPBYDAY/JFsvhbAYDAYDIakMErKYDAYDH6LUVIGg8Fg8FuMkjIYDAaD32KUlMFgMBj8FqOk\nDAaDweC3GCVlyHCIyAoRecbXchgM/oCIVBARh4hkyvd5pryptCAiISJyXkRy+loWXyEipUXkWCrL\n/CgiUSJy0T4uichmb8loyHyIyGERaeVrOVwhIn1E5LNUlnHY7eCi0993vCWjTaad8GqUFNaXCHAP\n4AAeSOdrZ0/P66VAB2BBGsoNUtVC9lFQVe/wtGAGg4+4H5ifyjIK1HNqD4VUdagXZMsSGCVl0RNY\nC0wAnnZOEJE8IjJMREJFJFxE/hSR3HbaPSLylx1/RER62vEJzFEi8pSIrHIKO0TkvyKyD9hnx30l\nIkdFJEJENojIPU75s4lIXxE5YH+VbRCRMiIyUkQSPPwiMltEXk98gyIySkSGJIqbJSJvOEV1wG6Q\nIvK+iBy3r7dbRFqm5ge164gzQzwvImH28bZTei77vsPsa41w7smKSGcR2Wz/JvtFpK1T9RVFZLUt\n30IRCbTL5BaRSSJy1v6/rBOR4qmV3eAf2M/Ofvv/OUtESjuljRCRf+3nY6uI3G7HdxCRnfazcUxE\n3nJRby77+bjdKa6YiESKSDE7XASoBqwVkaIiMtcuc05EViYntn24up9+IjJDRKba8v0jIvWc0mva\n749wEdkuIp2c0pJ8F9nXe8J+D50Wkb5O5Rrb74wIETmZ+J3h96hqlj+A/cCLQEPgOlDcKe07YDlQ\nCutBaArkBMoDF4GuQHYgAOvrCWAF8IxTHU8BfzqFHcAioDCQ247rARTB+nB4EzgJ5LLT3gW2AlXt\ncF37eo2B4071FgUuA8Vc3GNz4IhTuAhwBShph3MAZ4D8QHXgqFNaeaBSEr/dj8AnSaRVsO91MpAH\nqAOcBlrZ6Z8Aa2y5iwJ/AQPstCbABae8pYHqTr/vfqAKkNsOD7TTXgBm2/EC3AEU8PUzZo5k29/h\nuP9zovhW9jNZ325z3wAr7bS2wAagoB2u4fS8ngCa2eeFgQZJXHcs8KlT+L/AfKfwY8Bk+3wgMMpu\nn9mBu5O5HwdQOYm0fkAU8JBdz9vAIfs8h/1cv2+ft8R6x1Szyyb1LoprZ6OBXEA94BpQwy63Bnjc\nPs8HNPH1/zxVz4evBfD1gWXmiwIC7PAu4HX7XIBIoI6Lcn2AmUnU6Y6SapGCXOeBuvb5HqBjEvl2\nAq3t81eAecnUGQrcY58/Byx1SmsFLLHPqwCngNZAjhTk/BG4assbbv/90U6LazzVnPIPAn6wzw8A\n7ZzS2gKH7PP/AcOS+X37OoVfjnu5AL2A1XG/nTn8/yBpJTUW+NIpnN9uq+XtF/ge4E5AEpULBZ7H\nVmDJXLc1cMApvBp4win8k9PLfQDwO1DFjftxYH1gObeJNnZaP2CNU14BwoC77XfRiUR1/QJ8nMK7\nqAIQC5R2ilsHdLXPQ+zrFvX1/zothzH3Waa+xaoaboenYCkVgGJYX+SHXJQrBxy8hesedw6IyDsi\nssvuxocDhezrx13LlQxgNaQn7PMngEnJXHMa0N0+74HVw4kj3tSnqgeBN4D+wL8i8ouzmcUFQ1Q1\nUFUD7L+9nNKUhPd6BLjNPr8Nq8fmKi2l3/eU03kkUMA+n4TVS51qmxC/FP8a9zO4z21YzwQAqnoF\n64VfRlVXACOxehf/isj/RCTuGXgYayzpiG06a5pE/SuAvLY5rAJWj+13ABERoA2w0M47GOt5XGyb\n3d9PQfY7ErWJJU5p8c5JammRMPteb3NOszkClMF6F+Qh6fcAwL9O585t4lmsnuYe2/x9fwqy+xVZ\nWkmJSB4sc10L21Z7EuvlXF9E6gJnsbrNVVwUPwZUTaLqK1jd6jhKucgT741jjz+9CzxiP9QBWN38\nOLv2sSRkAPgZ6GzbtWsCs5LIB5YCfkREymN9gc50SotXUgCqOlVVm2N9pQF8mUy9ySFYCieO8ljm\nGOy/FZzSKjilJXfPSaKqMar6qarWBpoBnbA+RAwZjwTPh4jkxzILhwGo6khVbQTcjvUSfteO36iq\nDwLFsUy/011VrqoOO60H1sfbPFsRgmVKD1XVc3beK6r6jqpWwXKueiuFcVqXY1I28e3BVoZl7Xs9\ngdU+nClv329y76JkUdWDqtpDVYtjKdtfRSRvauvxFVlaSWHZhWOAWlhfUfXt89VAT/sr50dguFju\n2dlEpKk9uD8ZaC0ij4hIdhEJFJH6dr1bgC4ikldEqmJ9ySRHQSAaOGcP6H5sx8UxFvjUrgsRqSsi\nAQCqGgb8g9WDmKmqUUldRFW3AOfs+haq6kW7vopY41977XB1EWkpIrmwxuiuYpkw0sr/2b9FbSxz\n3FQ7fgrwkT1gXQz4P270BMcBvWw5RERuE5HqKV1IRIJEpI5Yc0YuY/2utyK7IX3IJZbTS9yRHev5\n6CUi9WwHgYHAWlU9KiKNRKSJiOTAej6vAQ4RySkiPUSkkKrGApewTGFJMQVr7KkHlmktjg7AH3EB\nEblfROIUxCWs90Zan6v/iMiD9j2+acv+N5aJ7oqIvCciOUQkCOgITLHfReNx/S6CZJSiiDwe5wwC\nRGB9IGecNuFre6MvDyx368Eu4h/F+qrJhtXFHo5lsgrHsu/GOTvcjfVwRWB1y5+044timZwigFVY\nNmXnMalYnAZW7euMs/OHAe9gdetbOaX3teMisB7m25zKP27Xea8b9/yRnbeLU9wrwDdO4br2NSKw\nvuDmAKWSqO9HrEZ20T4uAafttLgxqefs+zoBvO1UNjfwlR0fBozAdhax0ztjOYxcxPKCjLPrLyeJ\nMT+gG9ZYxSUs55MRQDZfP2vmSPaZPGw/k7H28xKL7YyD5QhzwOk5vM2Ob+X0bJzG+rjJh+VIsADr\nY+yC/RzflcL192M5aORwitsANHQKv2HLeQnLRN03mfrilONFp7/D7bR+WL23KXb8RqC+U9laWO+Y\nC8AO4AGntNy4eBdxY0wqm1Pe+DZi/zb/2tfbDnTy9f88NYfYN+EVRGQc1pfAv6paL4k83wDtsUxk\nT6v1tY+IBGO9wLIB41R1kNcEzeCISHNgkqpWTGP5P4BvVXVhiplTV28FLMWaUy3TiiGNpNQeRKQH\nllcYWC/G/6rqNjstFOuDwwFEq2qT9JI7IyIiJYBNqlrWC3X3w3K+MCZoN/G2ue9HoF1SiSLSHusf\nVg3LBfx/dnw2rEHRdkBtoLuI1PSyrBkSu7v/OvDDLVSzwj68QXK2eYMbuNkeDmH1pOsDnwFjnNIc\nQJCq3mEUlFsUxnINN/gBXlVSqroaq1uaFJ2xvNNQ1XVAYREpiTVHZr+qHlHVaKwxjM7elDUjYr+o\nwoGSwNdprUdVh2oyY1m3iPe66lmHFNuDqv6tqhF28G8sj7A4BDP+7Daqul9Vp/laDoNFDh9fvwwJ\nXS6P23Gu4s0XYCJUdQ833Ez9DlU9gjVJ0XBrpLY9PEfC5a0UWCIiscAYVb2VXrfhFlDVAb6WIaPh\nayWVmDSZhkTEfK0bvIqqZgizpe0W3QtrYmgcd6vqSbGWh1oiIrttK0fisqYdGbxOatuSr00AYSSc\nQ1PWjgsj4XyBuPgk8ZQnSb9+/UxdPqzr+HFlzRrX6fPmKTVrKrlyKblzK/fcowwcqMyfrxw+fCPf\nuXNK9+79iIlRzpxRKldWihRR5s5VYmOVb75RnnpK+fRTpW1bpUQJpXBh5Y47lMBApXp1pUAB5b77\nlIgIv3hvu9Ue7LlyY7A8wuLN7Kp60v57BmuyapK9MH99LjxVlz/LlhXqSgvp0ZNKcrFFLJfSV4Bp\n9qzwC6r6r4icBara3mEnsdyKuydRhyGDsGkT5MsHlStDrlw3p//7LzRtCpGRMHgwPGvPLrt2DZ55\nBjZuhK++glatIDoaVq+GRYusuC1b4PbbrbSRI8HhgMaNret07QqdO0OnTlC/Ply5YtV34AD06AFz\n5oAIrFsHlSpB2bJw+TKsXAkFC94spw/YQArtwZ6gPRNrGsRBp/h8WK7Jl+3JsG2xlvgxGDIEXlVS\nIvILEAQUFZGjWHMEcmGtBjJGVeeLtWLxASwX9F5YibEi8iqwmBsut7u9KavBe0RGwptvwh9/QN68\ncPo0DB0Kzz0Hs2fD2LEQEwOrVsH48XDHHRAcDMuXW8rl228txbF1K+TJY9WZO7eVJzjYCkdHw6+/\nwuLFluL6/XeoWRM2b4bPP4ds2azrbN4MH34IOV3sGta8+Y3zAgXgfj9ZPCap9iAiL2K3JayJ0IHA\nKHsVgzhX85LA77YpLwfWgqmLfXMnBkMa8GQ32leHdRueYcWKFaauJLh2TXXePNVZs1S3bbPili9f\noR9/rNqggeojj6i++KJ1xF3i2DHVevVUe/RQjYiw4nbtUq1bV7VqVdVatVR/+UV1wQLViRNvyHXm\njOrXX6u2a6f64YeqsbHpc4+usJ8vnz/n3j6yQjvydH2mrtSRlrbk1cm86YWIaGa4D39g2jSrB3Py\nJLz8MrRvb8VfuABdusClS3DbbZZp7JVXrB7MrFnw/fdw9ChERMDVqzBkCDz2GMycCb17wzvvWCa1\nOK5eta7Ttq3rXo0/ISJoBnGcuBVMOzJ4m7S0pUytpCpWrMiRI0dclDBUqFCB0NDQBHFr18LDD8NH\nH0H+/PDZZ1C+vHWsXWsplBEjIHt2CAuDJ56wxpFCQqBEiYT1nzoFr70GDz0E3TP4aGJWV1KmHSWP\nq7ZkcI1RUjfHkxnuzxuICG++qQwfboVjYuA//4EPPoBu3ay4qCirlxQZaSmhDh0S9oYcDqucKyeI\nzERWV1KmHSWP+X3cxyipm+PNw5MEIsJttymTJkHLlvDFF1aPaNGihIrIYJSUaUfJY34f90lLW/K3\nybyGdGTsWMsV+/bb4dAhy/vOKCiDweBPGCWVhWnfHnr1ssafZs3K/GY7g8GQ8TDmviyK+W3cx5j7\nzLOSHOb3cZ+0tCVfL4tkSCMvv/wyn3/+ua/FMBgyPKYt+TemJ+UjKlWqxLhx42jVqpVPru/Pv42/\nYXpS/v2smLaUcTA9qUxCbGysr0UwGDIFpi1lfIyS8gE9e/bk6NGjdOzYkUKFCjFkyBCyZcvG+PHj\nqVChAq1btwaga9eulC5dmoCAAIKCgti1a1d8Hb169eLjjz8GYOXKlZQrV47hw4dTsmRJypQpw4QJ\nE3xxawZDumLaUubHKCkf8NNPP1G+fHn++OMPLl68SNeuXQH4888/2bNnD4sWLQKgQ4cOHDx4kNOn\nT9OwYUMef/zxJOs8deoUly5d4sSJE4wdO5ZXXnmFiIiIJPMbDJkB05YyP1laSYl45kgrznZsEWHA\ngAHkzZuX3LlzA/D000+TL18+cubMyccff8zWrVu5dOmSy7py5crF//3f/5E9e3bat29PgQIF2Lt3\nb9qFMxhSgS/bEZi2lJnJ0kpK1TOHpyhbtmz8ucPhoE+fPlStWpUiRYpQqVIlRISzZ8+6LFu0aFGy\nZbvx78yXLx+XL1/2nHCGLEFMTNrK+VM7AtOWMhNZWkn5EnHx6egc98svvzB37lyWL1/OhQsXCA0N\ndd5SwWDwClev+lqC1GPaUubGKCkfUapUKQ4dOgTgssFcunSJ3LlzExAQwJUrV/jggw9cNkaDwZNk\nRCVl2lLmxigpH9GnTx8+/fRTAgMDmTlz5k2NpmfPnpQvX54yZcpQp04dmjVrlqr6TSM0pIXISF9L\nkHpMW8rcmMm8WRTz27hPVprMu2uXUqvWTfHmWUkG8/u4j5nMazAYbomMaO4zZG68rqREJFhE9ojI\nPhF530V6ERH5TUS2isjfInK7U1qoHb9ZRNZ7W1aDIauTEc19hsyNV7fqEJFswEigNXAC2CAis1V1\nj1O2vsBmVe0iIjWA74D77DQHEKSq4d6U02AwWJielMHf8HZPqgmwX1WPqGo0MBXonCjP7cByAFXd\nC1QUkeJ2mqSDjAaDwcYoKYO/4W0FUAY45hQ+bsc5sxXoAiAiTYDyQNxMPAWWiMgGEXney7IaDFke\nY+4z+Bv+sDPvl8DXIrIJ2A5sBuKWLr5bVU/aPaslIrJbVVf7SlCDIbNjelIGf8PbSioMq2cUR1k7\nLh5VvQQ8ExcWkcPAITvtpP33jIj8jmU+dKmk+vfvH38eFBREUFCQJ+Q3ZEFCQkIICQnxtRg+wSgp\ng7/h1XlSIpId2IvlOHESWA90V9XdTnkKA5GqGm2b9O5W1adFJB+QTVUvi0h+YDEwQFUXu7iOmSeV\nSsxv4z5ZaZ7UkCHKO+/cFG+elWQwv4/7pKUtebUnpaqxIvIqloLJBoxT1d0i8qKVrGOAWsBEEXEA\nO4Fn7eIlgd9FRG05J7tSUAaDwXOYnpTB3/C655yqLlTVGqpaTVW/tONG2woKVf3bTq+lqo+oaoQd\nf1hVG6jqHapaN65sZqFSpUosX778luqYOHEizZs395BEBkPGVFKmLWVujHt3BkZVzbpiBo+SVb37\nTFvyX4yS8gFxW1536tSJQoUKMXToUNatW8fdd99NQEAAd9xxBytXrozPP2HCBKpUqUKhQoWoUqUK\nU6ZMYc+ePbz88susXbuWggULEhgY6MM7MmQWMlpPyrSlLEDc0vYZ+bBu42aSivcHKlasqMuXL1dV\n1bCwMC1atKguXLhQVVWXLl2qRYsW1bNnz+qVK1e0UKFCun//flVVPXXqlO7atUtVVSdMmKDNmzdP\n0/X9+bfxN+zfyufPubcPQJ98Msn791tMW8o4pKUt+cM8KZ8hAzzTvdd+afPsUdsj6Oeff+b++++n\nXbt2ALRu3ZpGjRoxf/58Hn74YbJnz8727dspW7YsJUuWpGTJkh6R22BITFrNfZ5oS2ltR2DaUmYm\nSyupW2kUnuTIkSNMnz6duXPnAlaDi4mJoVWrVuTLl49p06YxZMgQnnnmGe655x6GDh1KjRo1fCy1\nITOSVnOfaUsGb2HGpHyE8yBtuXLl6NmzJ+fPn+f8+fOEh4dz6dIl3nvvPQDatGnD4sWLOXXqFDVq\n1OCFF164qQ6DwRNktDEpMG0ps2OUlI9w3vL6iSeeYO7cuSxevBiHw8G1a9dYuXIlJ06c4PTp08yZ\nM4fIyEhy5sxJgQIFyJbN+reVLFmS48ePEx0d7ctbMWQiMqJ3n2lLmZzUDmL540EGdJyYPXu2li9f\nXgMCAnTYsGG6fv16bdGihQYGBmqJEiW0Y8eOeuzYMT158qS2aNFCixQpogEBAdqyZUvdvXu3qqpe\nv35dO3bsqIGBgVq8ePFUXd+ffxt/gyzkOFGvXpL377eYtpRxSEtbMtvHZ1HMb+M+/rAskogEA19x\nY+WWQYnSewBxm4peAv6rqtvcKetUh1arpuzbd1O8eVaSwfw+7mO2jzcYMiFOm4e2A2oD3UWkZqJs\nh4B7VbU+8BkwJhVl48mI5j5D5sYoKYPB/0lx81C1lheLsIN/c2PfNnc2Ho0nIzpOGDI3RkkZDP6P\nO5uHOvMcsCAtZY2SMvgbWXqelMGQ2RCRlkAv4J60lL96tT/9+oGI2ZfNcOt4Ym824ziRRTG/jfv4\n2nFCRJoC/VU12A73wfKSSuw8UQ+YCQSr6sHUlLXTNHduJTwc8uZNEG+elWQwv4/7GMcJgyFzsgGo\nKiIVRCQX0A2Y45xBRMpjKagn4xSUu2WdyZvXmPwM/kWmNvdVqFDBzCRPggoVKvhaBIObqHubh/4f\nEAiMEuuhj1bVJkmVTeparpSUaUfJY9qSd8nU5j5PEx0N9erB0KFw//1ev5zBT/C1uS+9EBGtUkVZ\nsACqVfO1NIbMiDH3eZnRo6FsWejQwdeSGAzewZj7DP5Gpjb3eZKICPj0U1i61PJ8MhgyI0ZJGfwN\n05Nyk3HjoHVrqFvX15IYDN4jXz6z6oTBv/C6khKRYBHZIyL7ROR9F+lFROQ3EdkqIn+LyO3ulk0v\nHA747jt47TVfSWAwpA+mJ2XwN7yqpNxcN6wvsNlec+wp4JtUlE0XFiyAgAC4805fXN1gSD+MkjL4\nG97uSbmzbtjtwHIAVd0LVBSR4m6WTRdGjoTevc1YlCHzY8x9Bn/D20rKnXXDtgJdAESkCVAeKOtm\nWa8SGwsDB8KOHfDYY+l5ZYPBN5ielMHf8Afvvi+Br0VkE7Ad2AzEpraS/v37x597Ys0xVejY0fqq\nXLsW8uS5peoMGQhPrDeWUTFKyuBveFtJhWH1jOIoa8fFo6qXgGfiwiJyGGtvnHwplXXGWUl5gmPH\nYPNmOH4ccviDKjekG4k/cgYMGOA7YdIZY+4z+BveNve5s+ZYYRHJaZ8/D6xU1cvulPUmW7dCgwZG\nQRmyFqYnZfA3vPoKdnPNsVrARBFxADuBZ5Mr6015ndm6FerXT6+rGQz+Qd68cOaMr6UwGG7g9X6C\nqi4EaiSKG+10/nfi9OTKphdbt8JDD/niygaD78ifHw4f9rUUBsMNzIoTSWB6UoasSNmycPSor6Uw\nGG5glJQLrlyxHCZq+KQPZzD4jipV4NAhX0thMNzAKCkXbN8ONWsapwlD1qNyZQgNtZYCMxj8AaOk\nXGBMfYasSr58UKQIhCU52cNgSF+MknKBUVKGrIwx+Rn8CaOkXGCUlCErU6UKHDzoaykMBgujpBKx\nYgUcOAANG/paEoPBNxglZfAnjJJyYt8+6NYNpk6FwoV9LY3B4BsqVzZKyuA/GCVlowpdu1pbxLds\n6WtpDAbfYcakDP6EUVI2a9ZYa5Y9/7yvJTEYfIsx9xn8CaOkbMaMgRdeMBsbGgzFi8P163Dhgq8l\nMRiMkgIgPBxmz4annvK1JAaDb4m4FoGINS5lTH4Gf8AoKWDSJOjQAYoV87UkBoNvORhu2fmMyc/g\nLxglBfz8M/Tq5WspDAbfc/C8UVIG/yLLK6nTp2HvXmjRwteSGAy+J64nVb26NSXDYPA1WV5JLVkC\nrVpBrly+lsRg8D1xPamaNWHPHh8LYzBglBQLFkBwsK+lMBj8g7ieVJySUvWxQIYsT5ZWUg4HLF5s\nlJTBEEeckopzIjJbyRt8TZZWUps2WY2xQgVfS2Iw+AenLp8iKiYKEas3tXevryUyZHW8rqREJFhE\n9ojIPhF530V6IRGZIyJbRGS7iDztlBYqIltFZLOIrPe0bAsXml6UweBMuULlCL0QCphxKYN/4FUl\nJSLZgJFAO6A20F1EaibK9gqwU1UbAC2BYSIStyeuAwhS1TtUtYmn5Vu0yCgpg8GZKoFVOHD+AGCU\nlME/8HZPqgmwX1WPqGo0MBXonCiPAgXt84LAOVWNscPiLRkvXoQtW6B5c2/UbjBkTKoEVIkfl6pR\nw5j7DL7H20qqDHDMKXzcjnNmJHC7iJwAtgKvO6UpsERENoiIR5d+DQmBpk0hb15P1mowZGyqBVZj\nz1mr+2R6UgZ/IEfKWbxOO2CzqrYSkSpYSqmeql4G7lbVkyJS3I7fraqrXVXSv3//+POgoCCCgoKS\nvejixdC2raduwZCZCAkJISQkxNdi+IS2Vdoy4u8RjOwwksqVs3H8OFy7Bnny+FoyQ1ZF1IsTIUSk\nKdBfVYPtcB9AVXWQU555wBeq+pcdXga8r6r/JKqrH3BJVYe7uI6m9j6qV4fp06FBg9TelSGrISKo\nqk/XxxeRYOArLOvHOOc2ZKfXAH4EGgJ9nduJiIQCEVhjvNFJje/GtaPao2ozttNY7ip3F7VqwYwZ\nUKeOd+7LkLVIS1vytrlvA1BVRCqISC6gGzAnUZ4jwH0AIlISqA4cEpF8IlLAjs8PtAV2eEKo0FBr\nG4J69TxRm8HgXdx0QDoH9AaGuKgiVQ5IXW/vyvSd0wFrXCq1Jr/rsdfZcdojTdVg8K6SUtVY4FVg\nMbATmKqqu0XkRRF5wc72GdBMRLYBS4D3VPU8UBJYLSKbgb+Buaq62BNyLVkCbdpAtiw9S8zgS0Qk\nQETc/UxK0QFJVc+q6kYgxkX5VDkgPVr7UWbsmoFDHdx5J6xY4W5Ji4lbJtJiQguux15PXUGDwQVe\nf02r6kJVraGq1VT1SztutKqOsc9Pqmo7Va1nH1Ps+MOq2sD++qsbV9YTzJkD7dt7qjaDr1FV+of0\nJ8bh6v3sP4hIiD0vMBDYBPwgIjeZr13gjgNScqTKAen24rcTkDeAEWtHcKDyO0xb9Q+xse5fbNzm\nccQ6YllycEkqRDQYXJPl+hIXLsCff8IDD/haEoOn2H12NwNWDuCvo3/5WpSUKKyqF4EuwE+qeie2\nqdvL3K2qDYEOwCsick9KBV5t/Crz9s/jRMxOaDiO1S7dlW5m5+mdHLt4jAFBA5iyY8qtSW0w4B/e\nfenKrFnWqueFCvlakszLn0f+pFKRSpQrXC5drrf44GJyZc/F3H1zaVHRr/dcySEipYGuwIepKBcG\nlHcKl7Xj3EJVT9p/z4jI71jmwxS9ZPsF9aNIzSK0OfAYM2a4t53NuM3jeLr+03Sr041+If2IjI4k\nX8587opqyGR4xFNWVTP8Yd2GewQHq06Z4nZ2QzIsPbhUX5v/msY6YhPE1xlVR3vP752mOh0Oh0bF\nRKWqTIfJHbTPkj5a/dvqabpmStjPlyee00eBbcAoO1wZmOlGuezAAaACkAvYAtRKIm8/4G2ncD6g\ngH2eH/gLaJtE2ZvuPdYRq0UGBmrxymEaE5P87xQVE6XFBxfXA+cOqKpqm5/a6LQd01LzUxsyOWlp\nSyma+0Skt4gE3Joq9A/OnYM1a6BjR19Lkn5cuHaBDWEbvFL3yA0jmbx9Mu8sfifuJUfohVAOhx9m\nxq4ZxDpuHshQVRYeWBifPzFfrP6C4J/dX6sqKiaKVUdW8U6zd7h8/TJ7z/rvEgmqOkOtcdf/2uFD\nqvqwG+VH5CTtAAAgAElEQVRSdEASkZIicgx4E/hQRI7a3rG35ICUTbLRsnIL8tZawZo1yef958Q/\nlCtcjiqBVQDoVqcbM3fPdPdSBoNL3BmTKglsEJHp9mKxPp0vcivMmmVN4C1QwNeSpB9Td0zlpT9e\n8ni9F6MusvzwcjY8v4FFBxfxv3/+B8DcvXN5tPajlMxfklVHV6GqvL/kfXae3gnAj1t+pP3k9ozd\nNPamOk9fOc3wtcPZfXY368Os9YR/3vYz36z7Boc6uHL9Cm8vept/TtyYQrf2+FpqFa9F0XxF6Vit\nI3P3zU1S5rXH1tJ+cnvaT27PxC0Tb0pPSnF6ChEZbDtO5BSRZSJyRkSecKespuyA9K+qllPVIqoa\nqKrlVfWyesABqVWlVhSst5y//04+3/qw9dxZ5s74cMPSDeP/7wZDWklRSanqR0A1YBzwNLBfRAba\nq0NkKObMgYce8rUU6cuGsA1sPbWVy9cvAzBo9SCqflOV/iH9OX/1fLJld5/ZTc2RNYmKiQLAoY74\nnsqcvXO4t8K9VAqoxK+P/srHIR8TfjWcufvm0ql6Jx6r/RjTdkxj9MbRTN05leDJwaw6sor3l77P\nlIen0Hd53/jVtuP4dOWnPF73cd5r9h7D1g5jz9k9vLHwDX7e9jMdJneg0Q+N2HxqMz1m9iAyOhKw\nxqPaVG4DQKcanZi9d3a8svlu/Xe0mdQmvv6v131NvRL1eKHhC/Rd3pdfd/0anxYZHUmD0Q14fcHr\n8ffrBdqq5TjREQgFqgLveutinqJlxZaczr+CLVuSz7fhxAaalLkxDatqYFUOhR/CoQ4vS2jI1Lhr\nFwTqY8143wN8D2wGBqfWvuiNAzfGpK5eVS1USPXcuRSzpollh5bphrAN3qn8Fqg7qq4W+bKILj+0\nXFVVm45tql+t/Uo7T+ms/5333/h8DodD1x1fpx8t+0h3n9mt0bHR2mhMI839aW5dc3SNqqrO3zdf\nc3ySQxfsX6Adf+mok7ZOii///Jzn9aW5L2nBgQX1UtQlPXj+oAYOCtRig4vprtO7dPia4Up/dOCf\nA1VVddDqQVp3VF19fs7z+vyc5/W52c9p0UFF9fTl03rx2kUNHBSodUbV0e/Wf6fXY67rF6u+0Mnb\nJquqavdfu+tr81/TNUfXaLVvqmnI4RBVVY28Hql1R9XVHjN76LhN47TMsDJabHAxPXDugEbFRGmR\nL4voiYsnVFV104lNWnxwcZ2zZ46qqr6+4HXtMq2LPjT1IW04uqHuO7sv/t7w3JjUDvvvWCDYPt/q\nibo9JJ/LZ8jhcGjgFyW0WuNDLtPjqPpNVd15emeCuFJDS+nRC0eTLWfIOqSlLbnz4L4ObAQWYQ38\n5rTjswEHU3tBbxzuKKkFC1TvuSc1P6f7HI84roW/KKxPz3raOxdII5ejLmu+z/Np7/m99bOVn2n4\n1XAtMLCAXo2+qkcvHNXAQYEaeT1SHQ6HdpjcQat8XUVfmvuSFhtcTDtP6axtJ7XV/877rw79a6iq\nqr63+D3t9EsnLTa4mBb6opBGXIuIv1bYxTDN+1leDf45OD7u7nF364i1I+LDSw8u1ejYaFVVjYmN\n0Z+3/qyj/xkdf6w9tjY+b9+lfbXtpLY3OWWoqp6LPKe3DbtNa3xbQwetHqQxsTdG9K9cv6IvzHlB\nC39RWLee2qovz3tZB/45UJceXKqNxzROUM/aY2u13PBy2u3XbnrbsNv0XOQ5dTgcOnLdSC02uFi8\nUvSgkvrS/sjbDOQEigPrPFG3h+S76beO491FfZT3A/TJmb30XOTNX3rnIs9poS8KJfhfqKo2H99c\nlx1almS9hqyFt5TUAKBCEmkuPYzS+3BHSb36quoXX7j5SzoRHRutX//9tX4S8kmCnkMcDodDg38O\n1q4zurr0Lpu3d55ejrqc+gu7SawjVidvm3zTy0FV9c/QP7XJD0105q6Z2mFyB525a6a2m9QuPj34\n52CdtHWSzt4zW+uMqhNfx7ZT27TTL5306IWjOnnbZO0yrYuqqt75w526/NBynbd3nn607KObrvfd\n+u909p7Z8eHrMddv6b7iFJor4pRrUlyNvqqqqisOr9AG/2ugry94XT8J+eSmfGevnNUnfntCFx1Y\nlCB+88nN2mxcMz0fed5jSkqtZzUQyG6f5wNKeapuD8iW5O+pqlqryXFt/b9HtM+SPjelLdy/UFtO\naHlT/DOzntHR/4xOtl5D1sFbSqopUNApXAi4M7UX8uaRUuNyOFQrVlTdvj1Vv6eqWo2v2jfV9KNl\nH2nNkTXjexWqVm+g79K++p/R/9Fr0de08BeF9cyVM/HpK0NXqvQXl8rNlVJJzPWY6zpnzxx9bMZj\nOm/vPBf35dDe83sr/dGZu2belD70r6H66h+v6omLJzTgywB9Yc4LCeSfsXOGNh/fXGuNrOWyflXV\nw+GHtdTQUnop6pLm/zy/Rl6PTFFufyImNkZLDy2tgYMCdfPJzakqG6cEPdiTygm8BvxqH73jLBP+\ncKTUjnr2VP3i+8MaOChQz0eeT5D2Scgn+t7i924qM/DPgfrOoneSrdeQdUhLW3LHu+974LJT+LId\nl2HYtQscDqhdO/Vlp+2cxsuNXubTVp+y+InFfL3uawaEDODHzT9y36T7WHt8LXO7zyV3jtw0KdOE\ntcfWAnD5+mV6ze5Fx+odWXIo4fIwW09tpfiQ4qw5ltCnd9eZXWw8sRGwFum8/5f7+fTPTylVoBQf\nrfgo7kUCwJXrV/h4xceEhIbwQ6cfGLpm6E2ybzixgcZlGlO6YGmK5CnCLzt+oW2VG/uTPFDjAXaf\n3U2J/CXoUK2Dy/uvULgCgjB522TuKH0HeXNmrA24smfLziO3P0L+nPmpX7J+qsp6wZH1e+A/wCj7\naEgGaksNGkDYzop0qt6JketHJkhbf2J9AqeJOKoVrcaB8APpJaIhE+KOkkqwD4aqOshgK1WsWgWt\nW0Nq3znXY68ze+9sHq39KADlCpdj0ROLOBpxlJVHVtKpeieWPLmE0gVLA9CsXLN4xfPB0g+4t8K9\njGg3giUHlyRQMH2W9SG4ajAPTXuIVUdW8e26b2n8Q2PaTGpDpymd6LusL0/Pepr8ufKz9tm1DG83\nnOux11l+eDkOdfDO4ncoO6Ism09tZuETC+nVoBenLp+6Sek5e1s1K9eMArkKUKfEjT0XcmXPxagO\no/j+/u+TfCGLCM3KNWPwmsG0qODXqzkkyRtN32BEuxHeUDqppbGqPqWqy+2jF9DY10K5S/361m7W\nH9zzAd+u/5YzV84Altfn+jDXSqpqYFX2n9uf3qIaMhMpdbWA37BMFDnt43VgVmq7bN48SMFM8e67\nqgMH3hzfa1YvXbh/YZLl5u2dp/eMd9/bYtGBRXrvj/fq7jO7tdjgYvEDzJW+qqTb/7VsjcsOLdMq\nX1fRqJgo/WnLTxrwZYA+PvNxXXRgkcbExuipS6c0+OdgbfFjiwSmtXGbxmm7Se30zYVv6t3j7o73\nUovj23Xf6kNTH4oPn7lyRgt9USje8WD8pvH6wpwX3L4XZ4atGab0R5ccXJKm8hkdPGfu2wRUcQpX\nBjZ5om4PyZfs73D2rGrBgqqxsaofL/9Ya42spXvO7NEHpz6orSa2cjlGeCnqkub9LK9LBxhD1iMt\nbcmdB7cE1tYAp4F/gV+AEqm9kDePlBrXww/fvBTS6iOrlf7o539+nmS5J397Ur/5+5tk63bmwtUL\nmv/z/Nrxl446ePXg+PgX576ow9YM06iYKG04umGCpWKSGvxPHH8t+pqWHFJSa39X26V31eWoy1pr\nZC3tNauXbv93uzb5oYk+P+d5t2VPjrXH1mqOT3J41QHEn/GgkmoNHAVCgJVYc6VaeqJuD8mX4m9R\nrpzqAWvVIx28erBKf9FnZj2T7FJW3nBD/3XnrwmmUBgyBl5RUhnhSKlxNWyoum7djbDD4dC7xt6l\nd429S1+a+5LLMjGxMVr4i8I39VhSou6oulpueLl47zJVq0G1ndRWe8zsoZ2ndE7WKy051h5bm6w8\nl6Iu6VO/P6U5P8mpI9aOSPN1EhMTG6Pz9833SF0ZEU8pKasqcgP17CM3fuSE5I6S6tJFdfLkG+F9\nZ/el+Jw1H988fp6ep3hk+iOa7/N8WfbDKaOSlrbkztp9eUTkFREZJSLj445UWhV9yuHDUKkSHI04\nyvC1w3l78dtERkfybrN3OX7puMsyYZfCyJ8rf/x4k7s83eBpvmn/DXly5ImPa1WpFUsPLeVoxFGm\nPDwlzWMjTcs2TVaeArkKMOHBCUT0ieCNpm94bAwme7bstK9mNuDyBKoaparb7CMKmOFrmVLDvfda\nW93EUa1otRSfs6qBVdl/Pm3jUksPLaXZuGZxShSAGEcMyw4to2axmszZm3ijb0Nmwx3HiUlAKayt\nq1dibRNwyZtCeZILFyA6GooVg77L+rIi1NpmdOKDE6lQpALHIo65LHco/BCVAyqn+npv3fUWD9Z8\nMEFcQN4Axj8wnjnd5qSLd1xG88DL4vjcmyM1JFZS7lAtsBqrjq4iOjY6Pk5VeWDKA/y4+ceb8kdG\nRxLriGXjiY30mNmDA+cPsO3fbfHpG8I2ULZQWV6/83Wm7px6U1lvL8O04/QOv99gMzPhjpKqqqr/\nB1xR1YnA/cCdKZTxG+J6UddirvLH/j8Y22ksw9sNp36p+pQrVI5jF10rqYPnD1IlwHPLEz7V4CkC\n8maKxeQNnsW7q9p6mHr14MQJOH3a/TKP1XmMsIthVPmmClO2Wxshjt00ln3n9vHBsg+4cO1CfN7d\nZ3ZTelhp8g/MT9DEIEZ3HM1jtR9j4YGF8XkWHVxEcNVgHqz5ICGhIYRfDQesRY8bjm7IJys/cUuu\nyOhIQkJD3L8RrE0dG41pxO+7f09VuTh2n9nNrjO70lQ2q+KOkor7/LkgInWAwljOFBmCOCW14MAC\nGpZuSMkCJePTiuUrRmR0ZPxipc4cDPeskjJkXURkrojMcXHMBYr6Wr7UkD073H03bu/UC1A5oDLL\nn1rOjEdn8NGKj3jlj1esBX67/kqn6p347M/PALgWc41uM7sxtM1Qwt8P50DvAzxU6yGCqwaz8GBC\nJdWuSjsK5S7EfZXv4+t1XxMVE8XTs56mTok6fLv+W85Gnk1WpohrEQT/HEynKZ14e9HbLreVOXD+\nAAv2LyDiWgQA0bHR9JzVk8ZlGie72n5SqCpP/P4EHSZ3iK8zIxEZHUnfZX3ZdHJTul7XHSU1xt5P\n6iNgDrALGOTuBeztPfaIyD4Red9FeiG7wW4Rke0i8rS7ZVNi2aFl9N3VnnKVrzBt5zS63t418bUp\nW6isS5PfwfCD8fviGAy3yFBgmItjKNaW7hmKtJj8AO4seyfrn1vPoQuHeKvpW9QpUYfPWn3GhC0T\n6D2/N11ndKVqYFWea/gceXPmjf+gDKoYxD8n/uFS1CXCr4az8/RO7il/DwAf32tNaC8+pDgnLp1g\ncpfJPHr7owz5a0iSclyKukTLiS2pX7I+oa+HsvnUZh7/7fEE416hF0JpMaEFA1cPpMzwMtT9vi6t\nf2pNyfwlmdxlMgsOLLhJsV2LuUbYxbD4+WNgjal9uOxDHOpg1p5ZONRBcNVgei/onaR812Ov8+bC\nN+N7iK5QVYavHc57S95LIHdaCb8azp6ze26Kn7t3Lm8teosxG8fQaEwj/jnxDx1/6cih8EPxeTae\n2EiTH5qwMnRlfNzl65d5ce6LlB9Rno9XfEzYRbc3kr6Z5LwqsJRY19R6YyQqH7ejaE6sHUVrJsrz\nAfCFfV4MOIc1WTjFsk51uPQkCZoQpEU/rKc1P2urhb4olGDJojhaTmjpcv5PozGNEix4asi64EHv\nPn8+kmpHiVmzRrVBA7eyusX64+v1q7Vf6YCQATcttxRH64mtddbuWfrS3Jf00emP3pR+OPywXrx2\nUVVVj0Uc04AvA+LnJibm5Xkv65O/PRnvlXgt+po2+F+D+DUGz0ee11oja8VPP4mKidKNJzbqhM0T\n4t8h9b+vr6uOrIqvc/qO6Zr/8/xaemhpDfgyQKftmKZbT23V4oOLa/3v6+vrC17XuqPq6pw9c/Ry\n1GWt/m11Hbx6sEvPyMGrB2uBgQW0+6/dE8RfuHpBx28ar9N3TNeuM7pqozGNtMkPTfTtRW/fVM/F\naxd11u5ZLt9512Ou64i1I7TxmMbaf0V/Hf3PaC01tJSWGVYmwXqbYRfDtNjgYtpvRT/tMbOH/rTl\nJ1VVHblupFb/trpO3jZZf9v1mxYfXFwHhAzQEkNK6MfLP9Z3F7+rlb6qpL1m9dL1x9frK3+8oq/N\nf01V09aW3Hlw/0ltpU5lmwILnMJ9gPcT5ekDjLTPKwH73C3rlHbTP2JD2AYtN7yctmkfqY2Gd0iw\nOrczPX/vqeM3jb8pPuDLAD19+bTLMoashVFSCYmKUi1QQDU83K3sHmHIX0O0xJAS2uSHJglW30+K\nsRvHavHBxfWR6Y/oe4vf03cWvaN/H/tblx9armWGlblJGe48vVOLDiqqI9aO0DLDyuj7S95Ptv6P\nln0Uv1bhz1t/1lJDS+nWU1tVVXXLyS1abng5DRwUqFO2T9Hwq+Ha4H8NtPGYxvHK5ND5Q9p4TGPt\n9EunBPMej0cc16KDiurWU1u15sia+su2X1RVNeJahDYd21Tb/NRGH572sL618C29Gn1Vz0We07qj\n6mqHyR108OrB+knIJxr8c7AWHFhQ7xp7l5YdXlZXH1kdX//fx/7WmiNrartJ7XTe3nn66h+vapuf\n2ui64+s0aEKQTtl+Y0Lpo9Mf1Q+Xfejy/n/c/KN2mdZFa39XO/4jf/+5/frs7Gf1s5WfJVDgznhL\nSX0JvAOUw1rBORAIdKtyeBgY4xR+AvgmUZ4CwHLgBHARaO9uWac0VbXm8/y26zcNvxqu3X7tpsPW\nDNMaNVQ3b7se/5WVmA+XfagDQgYkiDsfeV4LDizosXlGhoyNUVI3c999qnPmuJ39ljlw7oA+MOWB\nJHtarrh47aKOWj9Kv1z1pfZb0U8rflVR83yWR+funesy/+h/RmvTsU0TvNSTYt3xdVrtm2r61O9P\nafkR5XXHvzsSpIddDEuwI8CFqxf05KWTCfJExUTpmwvf1CpfV9Ed/+7QIxeO6P2T79e+S/uqquo/\nYf9okS+LaPDPwdpwdEN9ed7LLt9J4VfD9Zdtv+hr81/Tdxe/qzN3zdTwq9YXxNy9c7XEkBLafHxz\n7TWrl5YcUlJn7Jzhsp5Zu2dpkx+aqMPh0Bk7Z2jlryt7fEHptLQlscoljYgcdhGtqpqif7aIPAy0\nU9UX7PATQBNVfS1Rnmaq+ra92+8SrImO7VIq61SH9uvXj52ndzJ//3yoBLmr5ObQa6GUDizE2bOQ\nP79rGUf/M5p/TvzDDw/8EB+38cRGnp3zLFteSmErUkOmJCQkhJCQkPjwgAEDUFWvuIqLSA5V9Qt/\nZpEEy3QmyxdfWB5+I0Z4WSgPEuOIYcfpHTQo1eCW63Kog9qjatO+ansGBA2gYO6Caa5r4paJvL7w\ndbJny073Ot0Z3GYw+XLmA+Bc5Dn+PPInZyPP8mzDZ8km7rgRJOTK9Sv8eeRPNp3cxDN3PJPkXMtY\nRyzVR1anTeU2zNk7h9ndZtO4jGeXlhSR1Lel1Gq11BxYJruFTmFX5r55wN1O4WVAI3fKOqWpw+HQ\nxmMa66zds/Rc5Dnd/u92PX5ctWTJ5DX7H/v+SLDHkqrqtB3T4vdQMhi4xZ4UsNrpfFKitAyzdp8z\nf/+tWq+e29kNKRB2McwvtsH534b/af3v62toeKhX6k9LW0pxNXMR6ZmEcvvJDR24AagqIhWAk0A3\noHuiPEeA+4C/RKQkUB04BES4UTae1UdXc+HaBTrV6EQ2yUZg3kBWrbLcz5PD1VwpT8+RMmR5nPvx\niTeMyVCTeeP4z38gNBTOnIHixX0tTcbntoK3+VoEAF74zwu88J8X/GHHgHjc2XLDub+XB2uRzE1A\nikpKVWNF5FVgMZa33jhV3S0iL1rJOgb4DJggInFTyt9T1fMArsomda2ha4fy1l1vJegO//UXNGqU\nvIyuXNAPhh+k0W0pFDQY3Cc5G1qGmswbR44c0Lw5hITAo4/6WhqDp/An5RRHikpKVRM49ItIEaxV\n0d1CVRcCNRLFjXY6P4k1/uRW2aRYcnAJUx6ekiBu4UJ4773kyxXJUwSHOoi4FkHhPIUBa0mkx2o/\n5s5lDQZ3KCIiD2F9bBURkS52vGBNjs+QtGoFy5cbJWXwLmnZvPAKlqu4X1GmUJn4wUaAixdh40Zo\nkcI+fSJCucKWyS9OSe0/v99M5DV4kpXAA07nnZzS0jAt1j9o1QpGj045n8FwK7gzJjWXGyaJbMDt\nwHRvCpUWyhUqlyC8bBncdVfSXn3OVA6ozL5z+6hTog7HLx7nWsw1KhXxOz1syKCotQNvpqNePTh7\nFsLCoEwZX0tjyKy405Ma6nQeAxxRVdf7W/iQcoUTKqmFC6G9m7tLBFUIYvnh5XSp1YWVoSu5t8K9\nfmmbNWQ+RKShqqbvYmgeIls2a1xq1Sro1s3X0hgyK+443R8F1qnqSlX9CzgnIhW9KlUacO5JqVpK\nKjjYvbJtqrRh8cHFAKw8spIWFVKwERoMnuNlXwtwK6R1HT+DwV3cUVIzAOcNWmLxw43anJXU/v2W\noqpZ072y9UrWIyIqgtALoUZJGdIVVX3e1zLcCkZJGbyNO0oqh6pejwvY57m8J1LacDb37dsHtWuD\nuxa7bJKN+yrfx09bf+LMlTPULVnXS1IasiIiUkFECjuFW4rI1yLyloj4XVtKDQ0awNGj1tiUweAN\n3FFSZ0QkzjMJEekM+N0j6dyTOnIEKlRIXfk2ldswbO0wmldonqalRwyGZJiOPaFXRBpgWSKOAvWB\nUT6U65bJkQOaNUvd/lIGQ2pwx3HiJWCyiIy0w8cBl6tQ+BLnnlRaldTFqIsEVQjyrGAGA+RV1RP2\n+RPAeFUdJiLZsLagydDEmfwefNDXkhgyIyl2GVT1oKo2xXI9v11Vm6nqAe+LljoK574xJzItSqpM\noTK0qNCCtlXaelgygyHB0ketsNanRFUdrrNnLFq0MONSBu+RopISkYEiUkRVL6vqZREJEJHP0kO4\n1ODsMp4WJQUQ8nQItUskXlrNYLhllovIdBH5GgjA2poGESkNXE+2ZAagUSM4cMBaFd1g8DTuDL60\nV9ULcQFVDcfPt7wODU2bkjIYvMQbwG9AKHCPqkbb8aWAD30llKfInRvatYM5c3wtiSEz4o6Syi4i\nueMCIpIXyJ1Mfp9y7RqEh0Np11umGAzpjr1LwVRVHaGqYU7xm1V1kS9l8xQPPQS//+5rKQyZEXeU\n1GRgmYg8KyLPYW1KONG7YqWdo0etJVqyZ/e1JAZDyojIGF/L4Ak6dLBWnrh40deSGDIb7jhODMLa\nTqMW1orkiwC/NaaldTzKYPARbi3RKiLBIrJHRPaJyPsu0muIyBoRuSYib6WmrCcoVMhaImn+fG/U\nbsjKuDsh6F+sRWYfxfJOSnJfJ19jlJQhI6GqG1PKY7uqj8Ta0qY20F1EEq+ncg7oDQxJQ1mP0KUL\n/PabN2o2ZGWSnCclItWxdsLtjjV5dxogqtoynWRLE0ZJGfwNEUnWpUBVH0guHWgC7FfVI3Z9U4HO\nwB6nOs4CZ0WkY2rLeooHHoC334ZLl6BgQU/XbsiqJDeZdw+wCugYNy9KRN5MF6lugSNHICjI11IY\nDAm4CzgGTAHWkfot48vY5eM4jqV8vF02VRQvbk3s/e03eOopb1zBkBVJztzXBTgJrBCRH0SkNalv\nXOmO6UkZ/JBSQF+gDvA10AY4a+8ssNKnknmYnj3hp598LYUhM5FkT0pVZwGzRCQ/lnngDaCEiHwP\n/K6qi9NJxlRx5AhUrOhrKQyGG6hqLLAQWGhP5+gOhIjIAFUdmXxpAMKA8k7hsnacO6SqbP/+/ePP\ng4KCCEqlWaJTJ3jxRcvLtnz5lPMbMjchISGEhITcUh2iqinnisssEoDlPPGYqrZ2s0ww8BVWr22c\n7S3onP4O8DiWY0ZOLC/CYqp6QURCgQisrUKiVdWlmUJEVFWJiYF8+eDyZciVodeWNvgTIoKq3pIV\nwVZO92MpqIrAHKw1/FJUNiKSHdgLtMaybqwHuqvqTQ5MItIPuKyqw9JQVlPzPkiKl1+GcuWgb99b\nrsqQyUhLW0qVkkottmfRPqwGcgLYAHRTVZeDtvag7xuqep8dPgT8x17lIrnrqKpy7BjceSecOJFc\nboMhddyqkhKRn7BMffOBqaq6Iw11BGOZCuM+9r4UkRex5gqPEZGSwD9AQayPustYa21edlU2iWt4\nREmtWwc9elhb5pj5igZn0tKW3FkF/VZIrWdRd6zB5TgE993kOX4cypZNo6QGg/d4ArgCvA685rTO\npGApmUIpVaCqC7HmKTrHjXY6/xcol7hcUmW9SZMmUKIEzJoFDz+cXlc1ZFa8vXGSK8+iMq4y2sst\nBQMznaIVWCIiG0QkxR1Mw8Ks1SYMBn9CVbOpakH7KOR0FHRHQWU0RODdd2HIEGuHbIPhVvB2Tyo1\ndAJWOy9mC9ytqidFpDiWstqtqi63V+vfvz9//w3nz0NISOoHfA2GODwx2JvV6dwZ3n8f/voL7rnH\n19IYMjLeHpNqCvRX1WA73AfLvDHIRd7fgOmqOjWJuvoBl1R1uIs0VVXeew8CAuCDDzx7H4asjScc\nJzICnhqTimPUKFi+HH791WNVGjI4aWlL3jb3bQCqikgFEckFdMPyakqAiBQGWgCzneLyiUgB+zw/\n0BZIdsDZmPsMBv+he3dYssTytjUY0opXlZQ9P+RVYDGwE8uzabeIvCgiLzhlfRBYpKpXneJKAqtF\nZDPwNzA3pblZYWHGccJg8BcCAqBZM7PorOHW8Kq5L72IM1NUqwbz5kGNdPNjMmQFjLkv7YwbB4sW\nwbUGHTkAABQVSURBVPTpHq3WkEHxu3lS6YWIqMOh5M9vbWFdoICvJTJkJoySSjtnz0KVKnDypDXR\n3pC18ccxqXTjwgXImdMoKIPBnyhWzJo3tXChryUxZFQyjZI6ftw4TRgM/sgjjxgPP0PayTRKynj2\nGQz+SefOsGABREX5WhJDRiRTKSnj2Wcw+B+lSkHdurB0qa8lMWREMpWSMj0pg8E/6dIFZs5MOZ/B\nkBijpAwGg9fp0gXmzIHoaF9LYshoGCVlMBi8TvnyUKkSzJ0LFy/6WhpDRsIoKYPBkC707g2vvgol\nS8JId/YjNhjIRJN5ixdXtm2zBmkNBk9iJvN6luXLra08Nm70+qUMfkaWXnEie3bl+nXIlmn6hgZ/\nwSgpzxITY1k91q6FypW9fjmDH5GlV5zIk8coKIMhI5AjBzz0EMyY4WtJDBmBTPNaz5vX1xIYDAZ3\n6drVKCmDe2QaJZUnj68lMBgM7nLvvXDsGBw65GtJDP5OplFSpidlMGQccuSARx+FiRN9LYnB38k0\nSsr0pAyGjMV//wtjxsD1676WxODPGCVlMBh8wu23Q+3aZmzKkDyZRkkZc5/BkPHo3Ru+/dY6zwSz\nYQxeINMoKdOTMhgyHh07wr//WvOmcueGTZt8LZHB3/C6khKRYBHZIyL7ROR9F+nviMhmEdkkIttF\nJEZEirhT1hmjpAyGjEf27PDXX7BmDXz4IYwd62uJDP6GV1ecEJFswD6gNXAC2AB0U9U9SeTvCLyh\nqvelpqyI6GOPKVOneulGDFkas+JE+nDsGDRoYO2ybcz3mRN/XHGiCbBfVY+oajQwFeicTP7uwJS0\nlDU9KYMhY1OuHDRuDLNm+VoSgz/hbSVVBjjmFD5ux92EiOQFgoG4rdHcLgtGSRkMmYFnnoHx430t\nhcGfyOFrAZzoBKxW1QtpKbxlS3/697fOg4KCCAoK8phghqxFSEgIISEhvhYjS9K5M7z+OvzxB9x/\nv6+lMfgD3lZSYUB5p3BZO84V3bhh6kttWVq2vKGkDIZbIfFHzoABA3wnTBYjd27L3NepE/z6q7V8\nkiFr421z3wagqohUEJFcWIpoTuJMIlIYaAHMTm3ZOIy5z2DIHNx5J0yZAg8/bHbxNXhZSalqLPAq\nsBjYCUxV1d0i8qKIvOCU9UFgkapeTalsUtcy3kAGQ+ahdWto3hymT/e1JAZfk2k2Pfz6a+W113wt\niSEzYlzQfcO8efD559bmiIbMgT+6oKcbxtxnMGQugoPhyBHYtcvXkhh8SaZRUsbcZzBkLnLkgJ49\n4ccffS2JwZf4kwv6LWF6UgZD5uPZZ60JvqGh8OSTltefZHrDq8GZTNOTMkrKkJlxZx1LEflGRPaL\nyBYRucMpPlREttprZK5PP6lvnWrVYP9+y/TXpw/06MH/t3fvQVKVZx7Hvw8gkhkUkTuibKKyWAjo\nJColGAWzCGqg1EoB3oCoIYvGNYmIyabKmFRtyR9oFuJtjLcoriYqK1ahXASMKCDXAOFiQFBBJBAB\ncQxEh2f/eM9IOzuXvp3p0z2/T9Wp6T59zsPbzTzz9DnnPe/L/qzupJRiVTJFSqf7pFRF41j+FrgE\n6AOMMbPetbYZDpzq7qcDE4AHU14+Alzk7me7+7lN1Oy86dQpHFGtWAEnnACDB2uixOakZIqUjqSk\nhKUzjuVI4PcA7r4MaGdmXaLXjBLI9bIyeOAB6NYNpkwpdGukqeialEjy1TWOZe0jotrb7IzW7QYc\nmGdm1UCluz8SY1tjZQYPPwwVFdClC8yeHe6p+tGPCt0yiUvJFCmd7hOp10B332VmnQjFaqO7L65r\nw1+mjC2W1DEwTz4Zpk6FRx4JwyY9/LCKVFLlYxzMkrmZd8sW59RTC90SKUWFvpnXzAYAv3T3YdHz\nOwF39ykp2zwELHT356Lnm4AL3X13rVh3AQfd/d46/p1E3cybjiNHoHv3MHGi8j/5dDOvSGlKZxzL\nWcD18GVR2+/uu82szMzaRuvLgaHA+qZrerxatAjd0mfVO6qnFLuSKVI63SelKp0xMN19NrDNzLYA\nDwMTo927AIvNbDWwFHjZ3ec2+ZuI0YgRKlKlrGRO91VVOWVlhW6JlKJCn+5rKsV4ug/gs8+ga9dw\nw++JJxa6NdIQne4TkWanrAyGDIGnny50SyQOJdO7r0XJlFsRydSvfgXDh0PLlnDzzYVujeRTyRQp\nEWm++vWDxYvhkkvC9envf7/QLZJ8KZlrUqXwPiSZdE2qeKxZEwrVhg3QoUOhWyO1ZZNLKlIijVCR\nKi633ALV1fDgg41vK01LRUokBipSxWXfPjjjDOjbN8xJde+94bkUnoqUSAxUpIrP1q2wZQusWhV6\n/b39NpSXF7pVksgiZWbDgN8Qurs/mjqUS8o2FwH3AccAe9x9cLR+O3CAMNXA5/VNM1BKySXJoyJV\nvNxh3Ljw+IknNGFioSWuSEXz4LwDXAx8SBjeZbS7b0rZph3wFjDU3XeaWUd33xu99i7wTXff18i/\nU3LJJcmhIlXcqqpg4EC44gq4665Ct6Z5yyaX4u6C/uU8OABmVjMPzqaUba4GXnD3nQA1BSpSEvPg\niEjhlJfD3Llw4YXQujX87GeFbpFkIu4CUNc8OCfV2qYXcKKZLTSz5WZ2XcprNfPgLDezm2Juq4iU\nqM6d4bXXwvWpcePg4EGYMydM91GCB48lJQk387YCKoAhQDmwxMyWuPsWMpgHR0SkId27w7JlMHEi\ndOwIZ54Jhw7B/v0waVKhWyf1ibtI7QROSXneI1qXagew190PAYfM7E9Af2CLu+8CcPc9ZjaTcPqw\naCdrk+KQj4naJJnatoUnnwzTz3frBjt2wIABcNpp4ZqVJE/cHSdaApsJHSd2AW8DY9x9Y8o2vYHp\nwDDgWGAZMArYDrRw90+jeXDmAnfXNc1AqV7wlWRQx4nStmIFXHYZ3HMPjB9f6NaUtsR1nHD3ajOr\nmQenpgv6RjObEF72SnffZGZzgLVANVDp7hvM7OvATDPzqJ0zSm0eHBEpvG99C15/HS6/HN5/Xz0A\nk0Y384o0QkdSzcOePaGr+u23ww9+UOjWlKbEHUmJiBSLTp1g9my44IIwOO1VVxW6RQI6khJplI6k\nmpfly2HUqNCh4ic/gVNOCQVMo1XkrlnPzCsikg/nnAPr1kHPnnDTTdC7dyhYc+cevafKHfbubTiO\n5IeOpEQaoSOp5u3IEXj+ebj77tBlvVcv2L4dPvkEHngAbrih0C0sHokbu6+pKLkkTipSUuPjj2Hz\n5nAK8OBB+Pa3YebMsG7GjHCv1dixcNxxhW5pMqlIicRARUrq8/LLoTCdfz788IehYC1bFq5rdelS\n6NYlj4qUSAxUpKQh770XjqxqOlb84hdh/qpXX4UWuur/FSpSIjFQkZJMfPEFDBkCffrA1VeHjhcd\nOqhggXr3iYgUXKtW8Nxz4chq0qTQ0aJNG6iogPvuC9PbS/p0JCXSCB1JSa4OH4Y334Tf/Q7WrIGF\nC49es/rnP+Gll8J1rZNqT2RUYnS6TyQGKlKST3ffDX/4A0yeDNu2wWOPhWlEtm4No7Nffz20bFno\nVsZDRUokBipSkk/u4bTfypXQo0foHThgQDjCmjgxDHI7ZgyMGAF9+4YehGvXhqLWsWOhW58bFSmR\nGKhISVPasAGeeQZeeQXWr4ehQ8NpwFmzYNq0UNRSj7RqrnG1b1+Y9mZCRUokBipSUijV1UcL0uuv\nhxHa9+yB734XysvDTcQLFoSeg1deGcYa7NOnsG1uiIqUSAxUpCRJVq4MBevw4TC78BVXhM4XlZUw\nfToMGgSjR4fehG3awKefwq5dsH8/XHopHHNM4dquIiUSAxUpKRZVVfDoozB/PqxeDZ9/Ho64unUL\nRa28PHTa6Ny5/hhffBGO3uIY9V33SYmINGPl5XDrreH61QcfwEcfhV6DixfD0qXhKKuiAh56KBSt\nGlVV8OKLcN550Lp1OH3Yvz889VRYP3ZsGEEDQsePysrQwaMp6EhKpBE6kpJSsnQp/PrX8MYbcPzx\n4Wjr4EE4+2z46U9h5MhwFDVvHkydGq6LXXxx6JG4YEG4p+uRR8L6mTPhH/+AJUtg/PhwhLZvH6xa\nBYMH//9RNnS6TyQGKlJSiv7+d/jss3Bqr2vXxodtmjEDbrstjPD+1lthufbaMKJGv34wZ04YBmrG\njNDTsF07uPHGo2MbTpig6eNFRCRNHTqEJV3XXBM6aAwaFIralVfCgQNHO2KsXAlPPhlG0zjjDHj2\nWZg9G04/PRSyrLl7rAswDNgEvANMrmebi4DVwHpgYSb7Rtt5vixcuFCxFOsrot+v2HOloSXNPJoG\n/BVYA5yVyb7eTPIo3/EUKzPZ5FKsHSfMrAXwW+ASoA8wxsx619qmHXA/cLm7nwl8L91947Bo0SLF\nUqxESTOPhgOnuvvpwATgoXT3jUOS/y+T2rbmECsbcffuOxf4q7u/5+6fA88CI2ttczXwgrvvBHD3\nvRnsK9IcpJMLI4HfA7j7MqCdmXVJc1+RxIq7SJ0EfJDyfEe0LlUv4EQzW2hmy83sugz2FWkO0smF\n+rZRHklxy/T8YCYLcBVQmfL8WmBarW2mA28BbYAOhPPmp6Wzb8prrkVLnEuceZKnPHoZOD/l+Xyg\nIp19lUdamnLJ9Pc/7t59O4FTUp73iNal2gHsdfdDwCEz+xPQP819AfBm0D1YmrV0cmEncHId27RO\nY19AeSTJFPfpvuXAaWbW08xaA6OBWbW2eQkYZGYtzawMOA/YmOa+Is1BOrkwC7gewMwGAPvdfXea\n+4okVqxHUu5ebWa3AHMJBfFRd99oZhPCy17p7pvMbA6wFqgmnJrYAFDXvnG2VySJ0syj2WZ2qZlt\nAaqA8Q3tW6C3IpKxkhhxQkRESlNRDzBrZsPMbJOZvWNmkzPc91Ez221ma1PWtTezuWa22czmRPdw\npROrh5ktMLO/mNk6M7s123hmdqyZLTOz1VGsu3JpW7RvCzNbZWaz8hBru5n9OWrf27nEM7N2ZvZH\nM9sYfXbnZfmZ9Yrasyr6ecDMbs2hXT82s/VmttbMZphZ61w+s2KgXEr7veYll5RHGcQqZK+lHHs8\ntQC2AD2BYwh32ffOYP9BwFnA2pR1U4A7oseTgXvSjNWV6A5/oC2wGeidQ7yy6GdLYCnhXpesYkXb\n/xh4GpiVy/uMtn8XaF9rXbbv8wlgfPS4FdAul7al/F58SOhEkHEsoHv0HltHz58DxubariQvyqWm\nzyXlUQaxmjIZ8rkAA4BXUp7fSQNDvtQTo2etxNoEdIkedwU2Zdm2/wW+k2s8oAxYAZyTbSxCb655\nhKGnahIr63YB24AOtdZlHA84Hthax/pcP7OhwBs5tKs78B7QPkr4Wfn4v0zyolxq+lxSHqUfq5hP\n98Vxk2JnDz2icPePgAamBqubmf0L4VvlUsJ/RsbxolMKq4GPgHnuvjzbWMB9wCTCPQo1so1FFGee\nhRuvb8wh3teBvWb2eHR6odJC785c2gYwCngm23a5+4fAVOB9QlftA+4+Pw/tSjLlUnrymUvKozRj\nFXORagoZ9Soxs7bA88B/uPundeyfVjx3P+LuZxO+uZ1rZn2yiWVmlwG73X0N0NA9MJm8z4HuXgFc\nCtxsZhdk0zbCt6sK4P4oXhXhG3xWnxmAmR0DjAD+WM++6XxmJxCGDepJ+DZYbmbX5NIuAZRLtSmP\n0oxVzEUq7Zt9M7DbwnhnmFlX4G/p7mhmrQhJ9ZS7v5RrPAB3/wRYRBjFOptYA4ERZvYu8D/AEDN7\nCvgo23a5+67o5x7CqZhzs2zbDuADd18RPX+BkGy5fGbDgZV+dPzHbGJ9B3jX3T9292pgJnB+ju1K\nOuVS4/KaS8qj9GMVc5HKx02Kxle/Fc0CxkWPxxJuNE7XY8AGd//vXOKZWceaHi9m9jXg3wg3N2cc\ny91/7u6nuPs3CJ/PAne/jjCETkaxovaURd9wMbNywnnrdVm2bTfwgZnVzDRzMfCXbGKlGEP4A1Ij\nm1jvAwPMrI2ZWdSuDTm2K+mUS43IZy4pjzKMlcnFtKQthG9Fmwlz6NyZ4b7PEHqvHI4+0PGEi3zz\no5hzgRPSjDWQcCPyGsK8WKuitp2YaTygb7T/GsINzv8Zrc84Vq24F3L0Ym9WsQjnv2ve47qazzyH\neP0JfyDXAC8SeiVlG6sM2AMcl7Iu21h3Ef6YrQWeJPR4y+nzT/qiXGq6XFIeZRZLN/OKiEhiFfPp\nPhERKXEqUiIiklgqUiIiklgqUiIiklgqUiIiklgqUiIiklgqUkXMzKpThtVfZWZ35DF2TzNbl694\nIkmlPEq2WGfmldhVeRivKy66iU6aA+VRgulIqrjVOdClmW0zsynRZGNLzewb0fqeZvaama0xs3lm\n1iNa39nMXozWrzazAVGoVtGoyuvN7FUzO7aJ3pdIU1IeJZiKVHH7Wq3TFN9LeW2fu/cD7gdqxkCb\nDjzu7mcRhrKZHq2fBiyK1lcQxv4COB2Y7u5nAgeAq2J+PyKFoDxKMA2LVMTM7BN3P76O9duAwe6+\nPRpRepe7dzKzPUBXd6+O1n/o7p3N7G/ASe7+eUqMnsBcd//X6PkdQCt3/68meXMiTUR5lGw6kipd\nXs/jTBxOeVyNrmFK86M8KjAVqeLW0ORro6Kfo4El0eM3CcPwA1wLvBE9ng9MhC9nMq35VtlQfJFS\noTxKMFX04tbGzFYRksCBV93959Fr7c3sz8AhjibUrcDjZnY7YTj+8dH624BKM7sB+AL4d8J02zoX\nLM2B8ijBdE2qBEXn0r/p7h8Xui0ixUp5lAw63Vea9M1DJHfKowTQkZSIiCSWjqRERCSxVKRERCSx\nVKRERCSxVKRERCSxVKRERCSx/g8tn5UdcyeFVgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fdc0ae73190>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_performance(history_convpool_5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "** Discussion**\n",
    "We obtain a classification accuracy of ~84 which seems at par with the accuracy obtained while training from scratch. The model converges in very few epochs."
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [Root]",
   "language": "python",
   "name": "Python [Root]"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
